Oct 30 18:34:40.381678 ip-10-0-132-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 18:34:40.381689 ip-10-0-132-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 18:34:40.381696 ip-10-0-132-168 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 18:34:40.381979 ip-10-0-132-168 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 18:34:50.471603 ip-10-0-132-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 18:34:50.471621 ip-10-0-132-168 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 993a3fea3e8c4d1eb98f4086d2397eee --
Oct 30 18:36:36.736727 ip-10-0-132-168 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:36:37.111380 ip-10-0-132-168 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:37.111380 ip-10-0-132-168 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:36:37.111380 ip-10-0-132-168 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:37.111380 ip-10-0-132-168 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:36:37.115069 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.114989    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.115069 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115069    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.115069 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115072    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115076    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115079    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115082    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115084    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115087    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115090    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115092    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115095    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115098    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115101    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115104    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115106    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115109    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115112    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115114    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115117    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115120    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115123    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.115159 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115126    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115136    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115139    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115142    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115144    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115147    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115150    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115152    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115155    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115157    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115160    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115163    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115166    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115169    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115172    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115174    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115177    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115179    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115182    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115185    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.115613 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115187    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115190    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115192    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115195    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115197    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115199    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115202    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115205    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115207    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115210    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115212    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115214    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115217    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115222    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115225    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115228    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115231    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115235    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115240    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.116083 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115245    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115248    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115252    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115254    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115257    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115260    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115263    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115265    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115268    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115271    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115273    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115281    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115284    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115287    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115290    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115292    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115295    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115297    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115300    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.116649 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115303    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115306    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115309    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115312    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115314    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115317    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115320    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115322    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115325    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115328    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115330    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115333    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115335    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115338    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115341    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115343    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115346    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115348    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115351    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115353    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.117095 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115356    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115358    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115361    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115364    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115368    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115371    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.115373    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116159    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116165    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116168    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116171    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116174    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116177    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116179    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116182    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116185    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116188    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116190    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116193    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116196    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.117592 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116198    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116201    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116204    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116207    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116210    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116213    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116215    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116218    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116220    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116223    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116225    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116228    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116231    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116234    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116237    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116239    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116242    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116245    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116247    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.118111 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116250    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116253    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116255    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116258    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116261    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116263    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116265    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116268    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116271    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116274    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116276    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116279    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116281    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116284    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116287    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116289    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116292    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116294    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116299    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116303    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.118583 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116306    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116309    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116312    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116314    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116317    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116320    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116322    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116325    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116328    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116331    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116333    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116336    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116338    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116341    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116344    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116346    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116349    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116353    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116357    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.119065 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116360    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116364    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116367    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116370    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116373    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116375    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116378    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116381    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116383    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116386    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116388    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116391    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116396    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116398    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116400    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116403    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116405    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116408    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116410    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.119537 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116413    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116415    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116418    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116421    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116425    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116427    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116430    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116432    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116436    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116439    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116441    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116444    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116463    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116466    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116469    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116471    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116954    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116959    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116962    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116965    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.119993 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116967    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116971    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116974    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116976    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116981    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116986    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116989    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116991    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116994    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116996    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.116999    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117002    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117004    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117007    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117010    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117012    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117016    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117019    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117021    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117024    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.120477 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117027    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117029    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117032    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117034    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117037    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117039    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117042    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117045    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117048    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117051    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117054    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117056    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117059    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117061    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117064    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117066    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117069    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117072    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117075    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117077    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.120943 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117080    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117082    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117085    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117089    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117092    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117095    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117098    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117101    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117105    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117107    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117110    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117113    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117115    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117118    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117120    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117122    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117125    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117128    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117130    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.121409 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117133    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117135    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117139    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117141    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117143    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117146    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117148    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117151    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117153    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117156    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117160    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117162    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117165    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117167    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117170    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117173    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117176    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117178    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117181    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.121866 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117184    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117186    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117192    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117194    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117197    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117199    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117202    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117204    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117207    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117209    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117212    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117214    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117217    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117220    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117223    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117226    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117229    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117231    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117233    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117236    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.122364 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117239    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117241    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117244    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.117248    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117337    2581 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117344    2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117352    2581 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117358    2581 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117365    2581 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117370    2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117377    2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117383    2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117386    2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117389    2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117394    2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117398    2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117401    2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117404    2581 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117407    2581 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117410    2581 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117413    2581 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117416    2581 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117421    2581 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117424    2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:36:37.122861 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117427    2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117431    2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117434    2581 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117438    2581 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117441    2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117456    2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117460    2581 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117463    2581 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117467    2581 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117470    2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117473    2581 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117477    2581 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117482    2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117485    2581 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117488    2581 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117491    2581 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117494    2581 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117497    2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117502    2581 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117505    2581 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117508    2581 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117511    2581 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117516    2581 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117520    2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:36:37.123420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117523    2581 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117526    2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117530    2581 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117533    2581 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117536    2581 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117539    2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117542    2581 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117545    2581 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117548    2581 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117551    2581 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117555    2581 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117558    2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117562    2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117565    2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117569    2581 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117572    2581 flags.go:64] FLAG: --help="false"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117575    2581 flags.go:64] FLAG: --hostname-override="ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117578    2581 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117581    2581 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117585    2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117589    2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117592    2581 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117596    2581 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:36:37.124056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117599    2581 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117601    2581 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117605    2581 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117608    2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117611    2581 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117614    2581 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117617    2581 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117620    2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117623    2581 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117626    2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117629    2581 flags.go:64] FLAG: --lock-file=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117632    2581 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117635    2581 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117638    2581 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117644    2581 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117647    2581 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117650    2581 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117653    2581 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117656    2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117659    2581 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117662    2581 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117665    2581 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117669    2581 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117672    2581 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117676    2581 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:36:37.124634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117679    2581 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117682    2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117685    2581 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117689    2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117692    2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117695    2581 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117698    2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117705    2581 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117708    2581 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117711    2581 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117714    2581 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117717    2581 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117719    2581 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117723    2581 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117726    2581 flags.go:64] FLAG: --port="10250"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117729    2581 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117732    2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0326a226477f2e310"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117735    2581 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117739    2581 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117742    2581 flags.go:64] FLAG: --register-node="true"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117745    2581 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117749    2581 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117752    2581 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117754    2581 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117758    2581 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117762    2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:36:37.125223 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117765    2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117768    2581 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117771    2581 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117774    2581 flags.go:64] FLAG: --runonce="false"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117777    2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117780    2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117784    2581 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117787    2581 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117790    2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117794    2581 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117798    2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117801    2581 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117804    2581 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117807    2581 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117810    2581 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117813    2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117816    2581 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117819    2581 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117822    2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117828    2581 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117831    2581 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117834    2581 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117838    2581 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117841    2581 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117844    2581 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:36:37.125885 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117847    2581 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117850    2581 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117853    2581 flags.go:64] FLAG: --v="2"
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117858    2581 flags.go:64] FLAG: --version="false"
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117862    2581 flags.go:64] FLAG: --vmodule=""
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117866    2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.117869    2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118009    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118013    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118016    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118018    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118021    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118024    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118027    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118030    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118033    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118036    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118039    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118041    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118044    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118046    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.126505 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118049    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118051    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118054    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118057    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118059    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118062    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118065    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118068    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118070    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118073    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118075    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118078    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118080    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118083    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118085    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118088    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118091    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118094    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118099    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.127010 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118101    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118104    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118106    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118109    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118111    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118114    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118117    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118120    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118123    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118126    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118128    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118131    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118133    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118136    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118139    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118141    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118144    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118146    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118158    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.127466 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118162    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118165    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118168    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118172    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118174    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118177    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118180    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118182    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118185    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118187    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118190    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118193    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118195    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118198    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118200    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118203    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118205    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118208    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118210    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118213    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.127928 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118215    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118218    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118221    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118224    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118226    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118229    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118231    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118234    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118237    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118239    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118242    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118246    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118249    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118252    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118254    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118258    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118261    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118263    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118266    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.128401 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118268    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118271    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118274    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118276    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118279    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118281    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118284    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118287    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118289    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118292    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118294    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118297    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118300    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118304    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118307    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.128870 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.118334    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118374    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118378    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118381    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118385    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118387    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118391    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118393    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118396    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118398    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118403    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118405    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118408    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118412    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118415    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118417    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118420    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118423    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118425    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118428    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.129228 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118430    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118433    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118436    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118438    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118441    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118443    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118457    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118460    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118462    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118465    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118467    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118470    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118473    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118476    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118479    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118482    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118484    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118487    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118490    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118493    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.129721 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118495    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118498    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118502    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118504    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118507    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118511    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118514    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118516    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118519    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118522    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118524    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118527    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118529    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118532    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118534    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118537    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118540    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118544    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118547    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.130199 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118550    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118553    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118555    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118559    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118561    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118564    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118568    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118571    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118574    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118576    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118579    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118581    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118584    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118587    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118590    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118594    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118597    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118599    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118604    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118607    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.130679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118609    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118612    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118615    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118618    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118620    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118623    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118625    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118628    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118630    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118633    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118636    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118640    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118643    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118646    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118649    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118651    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118654    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118656    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118659    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.131178 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118661    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118665    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118668    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118670    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118673    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118675    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118678    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118680    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.118685    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.126090    2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.126107    2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.126118    2581 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.126124    2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126195    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126202    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126205    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126208    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126211    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126214    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126216    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.131643 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126219    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126222    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126224    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126227    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126230    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126232    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126235    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126237    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126240    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126242    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126245    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126248    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126250    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126253    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126255    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126258    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126260    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126263    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126266    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126269    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.132113 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126272    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126274    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126283    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126286    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126289    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126292    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126294    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126297    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126300    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126302    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126305    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126307    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126310    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126312    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126315    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126318    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126320    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126323    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126325    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126328    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.132603 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126330    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126333    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126336    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126338    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126341    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126344    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126346    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126349    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126351    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126354    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126357    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126359    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126362    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126364    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126367    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126376    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126379    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126381    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126384    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126386    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.133068 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126389    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126393    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126396    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126399    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126402    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126404    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126407    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126409    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126412    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126414    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126417    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126419    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126422    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126424    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126427    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126429    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126432    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126435    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126437    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.133701 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126440    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126444    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126466    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126471    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126475    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126480    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126483    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126486    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126489    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126498    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126502    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126505    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126508    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126511    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126514    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126516    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126519    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126522    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126525    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.134176 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126527    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.126556    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126742    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126747    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126750    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126753    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126755    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126758    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126761    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126763    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126766    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126769    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126771    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126774    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126776    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126779    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.134906 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126782    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126785    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126787    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126790    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126793    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126796    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126798    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126807    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126810    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126813    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126816    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126819    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126822    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126824    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126827    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126829    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126832    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126834    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126837    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126840    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.135585 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126842    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126845    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126847    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126850    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126852    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126855    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126858    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126860    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126863    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126865    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126868    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126870    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126873    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126876    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126885    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126888    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126891    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126894    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126896    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126899    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.136257 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126907    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126910    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126913    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126916    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126919    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126921    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126924    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126926    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126929    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126931    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126934    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126936    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126939    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126941    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126944    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126947    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126949    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126951    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126954    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.136843 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126956    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126959    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126961    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126964    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126966    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126969    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126971    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126974    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126976    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126979    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126981    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126984    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126986    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.126991    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127000    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127003    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127006    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127009    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127011    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.137288 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127014    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127016    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127019    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127022    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127024    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127027    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127029    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127032    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127036    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127039    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127042    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127045    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127047    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127050    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.127073    2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127109    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:36:37.137884 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127114    2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127116    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127119    2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127122    2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127125    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127127    2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127130    2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127133    2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127135    2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127138    2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127141    2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127144    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127153    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127156    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127158    2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127161    2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127164    2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127166    2581 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127169    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:36:37.138383 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127171    2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127173    2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127176    2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127180    2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127184    2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127187    2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127190    2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127192    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127195    2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127197    2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127200    2581 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127203    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127206    2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127208    2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127211    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127214    2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127216    2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127220    2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127223    2581 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:36:37.138854 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127226    2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127229    2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127231    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127234    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127236    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127239    2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127242    2581 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127250    2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127253    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127255    2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127258    2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127260    2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127263    2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127265    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127269    2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127271    2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127274    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127276    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127279    2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:36:37.139328 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127281    2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127284    2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127286    2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127289    2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127291    2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127297    2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127299    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127302    2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127305    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127307    2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127310    2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127313    2581 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127315    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127318    2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127320    2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127323    2581 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127325    2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127328    2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127330    2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127333    2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:36:37.139891 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127335    2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127344    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127355    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127358    2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127360    2581 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127363    2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127365    2581 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127368    2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127370    2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127373    2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127375    2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127378    2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127380    2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127383    2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127385    2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127388    2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127390    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127394    2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127397    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127400    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:36:37.140509 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127402    2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127405    2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127408    2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127410    2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127413    2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127415    2581 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127418    2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:37.127420    2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.127504    2581 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.130083    2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.131203    2581 server.go:1014] "Starting client certificate rotation"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.131304    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.131343    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.133677    2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:37.141000 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.138115    2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:36:37.150819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.150795    2581 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:36:37.155854 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.155833    2581 log.go:25] "Validated CRI v1 image API"
Oct 30 18:36:37.156686 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.156660    2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:36:37.160535 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.160513    2581 fs.go:135] Filesystem UUIDs: map[09cf16da-8ed1-49a7-bc4e-05b1176fba13:/dev/nvme0n1p3 64eb4ad3-b8fc-4c78-aef2-c65c38169ad6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 18:36:37.160604 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.160533    2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:36:37.165886 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.165772    2581 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:37.164239204 +0000 UTC m=+0.347873986 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3180463 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2545cafeddc65595ea17506f692915 SystemUUID:ec2545ca-fedd-c655-95ea-17506f692915 BootID:993a3fea-3e8c-4d1e-b98f-4086d2397eee Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:de:81:19:2d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:de:81:19:2d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:8b:de:90:a6:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:36:37.165886 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.165885    2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:36:37.166014 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.166002    2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:36:37.169625 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169602    2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:36:37.169771 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169628    2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:36:37.169816 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169776    2581 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:36:37.169816 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169785    2581 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:36:37.169816 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169795    2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:37.169816 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.169806    2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:36:37.171123 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.171110    2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:36:37.176030 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.176017    2581 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:36:37.176162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.176152    2581 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:36:37.177025 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177014    2581 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:36:37.177056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177032    2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:36:37.177056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177043    2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:36:37.177056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177052    2581 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:36:37.177135 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177097    2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:36:37.177602 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.177583    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:37.180283 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.180262    2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:36:37.187593 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.187580    2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:36:37.188599 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188586    2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:36:37.188633 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188617    2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:36:37.188791 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188780    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:36:37.188819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188798    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:36:37.188819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188804    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:36:37.188819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188809    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:36:37.188819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188814    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:36:37.188819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.188821    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:36:37.189459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189439    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:36:37.189489 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189460    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:36:37.189489 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189468    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:36:37.189489 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189474    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:36:37.189489 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189488    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:36:37.189609 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.189497    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:36:37.190412 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.190401    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:36:37.190460 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.190415    2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:36:37.195332 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.195313    2581 server.go:1263] "Started kubelet"
Oct 30 18:36:37.195462 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.195401    2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:36:37.195532 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.195472    2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:36:37.195584 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.195552    2581 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:36:37.196098 ip-10-0-132-168 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:36:37.196934 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.196905    2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:36:37.197676 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.197663    2581 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:36:37.201851 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.201833    2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:36:37.207065 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207051    2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:36:37.207124 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207065    2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:37.207411 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.207165    2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found"
Oct 30 18:36:37.207839 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207822    2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:36:37.207839 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207820    2581 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:36:37.207956 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207848    2581 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:36:37.207956 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207938    2581 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:36:37.207956 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.207948    2581 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:36:37.208208 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208184    2581 factory.go:55] Registering systemd factory
Oct 30 18:36:37.208208 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208204    2581 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:36:37.208601 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208576    2581 factory.go:153] Registering CRI-O factory
Oct 30 18:36:37.208601 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208593    2581 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:36:37.208728 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208646    2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:36:37.208728 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208666    2581 factory.go:103] Registering Raw factory
Oct 30 18:36:37.208728 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.208680    2581 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:36:37.209747 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.209734    2581 manager.go:319] Starting recovery of all containers
Oct 30 18:36:37.219024 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.219000    2581 manager.go:324] Recovery completed
Oct 30 18:36:37.223959 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.223937    2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:36:37.224852 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.223882    2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:36:37.225400 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.223918    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-168.ec2.internal.18e3628459327ffe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-168.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-168.ec2.internal,},FirstTimestamp:2026-10-30 18:36:37.195276286 +0000 UTC m=+0.378911049,LastTimestamp:2026-10-30 18:36:37.195276286 +0000 UTC m=+0.378911049,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-168.ec2.internal,}"
Oct 30 18:36:37.225658 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.225643    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.227295 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.227279    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.227379 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.227314    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.227379 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.227329    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.228190 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.227994    2581 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:36:37.228190 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.228004    2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:36:37.228190 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.228022    2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:36:37.229039 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.229025    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.229983 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.229960    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.230115 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.229994    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.230115 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.230005    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.230359 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.230342    2581 policy_none.go:50] "Start"
Oct 30 18:36:37.230359 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.230360    2581 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:36:37.230475 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.230370    2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:36:37.232421 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.232410    2581 policy_none.go:44] "Start"
Oct 30 18:36:37.233463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.233436    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:36:37.233553 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.233510    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z2v28"
Oct 30 18:36:37.242643 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.242572    2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-168.ec2.internal.18e362845b1b16dd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-168.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-168.ec2.internal,},FirstTimestamp:2026-10-30 18:36:37.227296477 +0000 UTC m=+0.410931248,LastTimestamp:2026-10-30 18:36:37.227296477 +0000 UTC m=+0.410931248,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-168.ec2.internal,}"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.267081    2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.271475    2581 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.272113    2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272126    2581 server.go:87] "Starting device plugin registration server"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272330    2581 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272342    2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272426    2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272518    2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.272527    2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.273063    2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.273107    2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-168.ec2.internal\" not found"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.274006    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.274596    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.275786    2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.275879    2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.276334    2581 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.276359    2581 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.276388    2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.298518    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:37.371999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.300463    2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:37.374513 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.373880    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.374907 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.374891    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.374951 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.374924    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.374951 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.374935    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.375008 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.374959    2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.377080 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.377056    2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"]
Oct 30 18:36:37.377122 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.377118    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.377886 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.377865    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.377944 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.377892    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.377944 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.377904    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.380071 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380057    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.380199 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380185    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.380286 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380214    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.380772 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380756    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.380865 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380773    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.380865 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380784    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.380865 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380797    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.380865 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380805    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.380865 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.380818    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.382868 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.382852    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.382940 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.382877    2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:36:37.383567 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.383552    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:36:37.383621 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.383577    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:36:37.383621 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.383590    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:36:37.392998 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.392981    2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.408244 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.408224    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.409371 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.409345    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.409431 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.409380    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.409431 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.409397    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.424076 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.424061    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.426089 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.426075    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510171 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510140    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510287 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510234    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510287 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510217    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510351 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510282    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510351 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510323    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.510351 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.510333    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.526536 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.526513    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:37.527001 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.526988    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:37.527093 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.527083    2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.527141 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.527129    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:37.527181 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.527163    2581 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.532928 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.532905    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.533590 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.533577    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:37.549808 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:37.549791    2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:36:37.549890 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:37.549834    2581 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:36:39.156800 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.156585    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ce239b215936dbe4b6418b6fb4e9c3.slice/crio-eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d WatchSource:0}: Error finding container eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d: Status 404 returned error can't find the container with id eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d
Oct 30 18:36:39.157037 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.157019    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35ca846eedff7c0a375d6f9a1bb598c.slice/crio-e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f WatchSource:0}: Error finding container e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f: Status 404 returned error can't find the container with id e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f
Oct 30 18:36:39.163298 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.163277    2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:36:39.221834 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.221812    2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:36:39.222342 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.221995    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:36:39.222342 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.222002    2581 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:36:39.268340 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.268317    2581 apiserver.go:52] "Watching apiserver"
Oct 30 18:36:39.298172 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.298118    2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:36:39.315067 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.315049    2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:36:39.315373 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.315357    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-mlgx9","openshift-network-diagnostics/network-check-target-kps9t","openshift-network-operator/iptables-alerter-8tlpv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2","openshift-cluster-node-tuning-operator/tuned-rwlfm","openshift-image-registry/node-ca-ghjt5","kube-system/global-pull-secret-syncer-kp5w5","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal","openshift-multus/multus-additional-cni-plugins-n9ff9","openshift-multus/multus-l9qs4","kube-system/konnectivity-agent-6bz4h","openshift-dns/node-resolver-j4266","openshift-ovn-kubernetes/ovnkube-node-k5ctz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"]
Oct 30 18:36:39.319156 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.319140    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.319234 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.319211    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:39.321224 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.321206    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:39.321313 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.321292    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:39.323365 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.323347    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.325503 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.325486    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.327618 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.327600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.329809 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.329758    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:38 +0000 UTC" deadline="2028-06-27 06:55:01.092867593 +0000 UTC"
Oct 30 18:36:39.329809 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.329804    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14532h18m21.763067778s"
Oct 30 18:36:39.329953 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.329816    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.332097 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.332083    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.332162 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.332146    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:39.334310 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.334291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.336460 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.336444    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.338796 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.338777    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.339428 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.339409    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.339581 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.339568    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.339898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.339882    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:36:39.339970 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.339882    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-84qrk\""
Oct 30 18:36:39.341129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.341101    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.343745 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.343730    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.344343 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.344326    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dzmp9\""
Oct 30 18:36:39.361869 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.361843    2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:36:39.361972 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.361906    2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dvfgj"
Oct 30 18:36:39.370715 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.370654    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerStarted","Data":"eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d"}
Oct 30 18:36:39.371805 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.371783    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"d35ca846eedff7c0a375d6f9a1bb598c","Type":"ContainerStarted","Data":"e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f"}
Oct 30 18:36:39.399439 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.399423    2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:36:39.404636 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404620    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.404699 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404643    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.404699 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404659    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.404699 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404679    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.404837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404758    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.404837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404826    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvhz7\" (UniqueName: \"kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.404930 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404862    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.404991 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404927    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.404991 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.404975    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405090 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405008    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.405090 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw5zx\" (UniqueName: \"kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.405090 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405068    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.405195 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405107    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405195 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405140    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405195 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405162    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405195 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405177    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405330 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405211    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxfsz\" (UniqueName: \"kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.405330 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.405330 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405271    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.405330 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405318    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.405440 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405408    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405440 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405424    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405516 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405438    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405516 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405453    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.405516 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405468    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snl4j\" (UniqueName: \"kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.405516 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405483    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.405516 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405510    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405565    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405582    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405605    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405621    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405657    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.405705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405683    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405711    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405751    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405784    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405803    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt6qt\" (UniqueName: \"kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405827    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405844    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405880    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405919    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405945    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.405961 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405960    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.405976    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406037    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxbmh\" (UniqueName: \"kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406099    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406133    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406191    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406215    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406233    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406261    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406286    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406319    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406370 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406355    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406375    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406404    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406431    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406462    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56cnn\" (UniqueName: \"kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406490    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjw94\" (UniqueName: \"kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406538    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406554    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406575    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406589    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406619    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406634    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.406726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406674    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406762    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406787    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406819    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406837    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406863    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwzn2\" (UniqueName: \"kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406896    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406936    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406958    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.406972    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.407009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.407050    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.407226 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.407117    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:39.440408 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440385    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:36:39.440517 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440465    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.440517 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440505    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440625 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440531    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:36:39.440625 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440545    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:36:39.440716 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440687    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440767 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440722    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440767 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440732    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rbclc\""
Oct 30 18:36:39.440767 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440730    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.440767 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440746    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440767 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440758    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440959 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440778    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.440959 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440782    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.440959 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440793    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:36:39.440959 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.440789    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:36:39.441191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441066    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:36:39.441191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441109    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:36:39.441191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441132    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.441191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441136    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:36:39.441191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441159    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:36:39.441380 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441213    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:36:39.441380 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.441221    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:36:39.454112 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454096    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:36:39.454455 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454440    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-d56w6\""
Oct 30 18:36:39.454524 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454492    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:36:39.454524 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454510    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r66x4\""
Oct 30 18:36:39.454603 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454533    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vr4rx\""
Oct 30 18:36:39.454817 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454799    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7kr8g\""
Oct 30 18:36:39.454898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454805    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-t7f6r\""
Oct 30 18:36:39.454898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454835    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h68zm\""
Oct 30 18:36:39.454898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.454860    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:36:39.508035 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508142 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508044    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508142 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508068    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.508142 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508090    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.508142 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508113    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508137    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508136    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508164    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508195    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508223    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508224    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508243    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508279    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt6qt\" (UniqueName: \"kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508290    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508289    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508305    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508312    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508147    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508330    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508355    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508357    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.508463 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508400    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508401    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508443    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508466    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.508487    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508560    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.508574    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:36:40.008529399 +0000 UTC m=+2.101365325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508488    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508612    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbmh\" (UniqueName: \"kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508623    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508634    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508578    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508651    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508784    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508811    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509129 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508852    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508855    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508866    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508882    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508833    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508899    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508931    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508943    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508970    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508990    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.508994    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509016    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509028    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509043    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509053    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509065    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56cnn\" (UniqueName: \"kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509058    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.509884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509093    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509132    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjw94\" (UniqueName: \"kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509165    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509095    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509207    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509260    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509291    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509346    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509355    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509380    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509264    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509382    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509401    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509436    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509437    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.510713 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509464    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509475    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509498    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509515    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509531    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509556    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzn2\" (UniqueName: \"kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509580    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509592    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509604    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509630    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509653    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509659    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509730    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509667    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509835    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510092    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.509770    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.511514 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510267    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510295    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510321    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510346    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510381    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510404    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510434    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhz7\" (UniqueName: \"kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510460    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510522    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510554    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510565    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510567    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510595    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510578    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5zx\" (UniqueName: \"kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510614    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510661    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510674    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510687    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.510703    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510736    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.510758    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:36:40.010739864 +0000 UTC m=+2.103575792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510787    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510649    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510703    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510804    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510831    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510852    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510855    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510903    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxfsz\" (UniqueName: \"kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510930    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510952    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510858    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.512979 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.510976    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511012    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511042    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511056    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511098    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511124    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snl4j\" (UniqueName: \"kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511149    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511156    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511176    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511207    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511220    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511334    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511348    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511015    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511402    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.513769 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511433    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511649    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511786    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.511944    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.512192    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.512335    2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.512419    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.514051    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.514646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.514090    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.516461 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.516435    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.516612 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.516444    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.516612 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.516603    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.516694 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.516640    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.540176 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.539947    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbmh\" (UniqueName: \"kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.553526 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.553468    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5zx\" (UniqueName: \"kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.553693 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.553677    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:39.553743 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.553695    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:39.553743 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.553705    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:39.553838 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:39.553765    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:40.053747283 +0000 UTC m=+2.146583207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:39.554765 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.554340    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhz7\" (UniqueName: \"kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.556347 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.556300    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt6qt\" (UniqueName: \"kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.557182 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.557160    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxfsz\" (UniqueName: \"kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:39.558169 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.558139    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snl4j\" (UniqueName: \"kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.558657 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.558630    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzn2\" (UniqueName: \"kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.559432 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.559408    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56cnn\" (UniqueName: \"kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.559532 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.559454    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjw94\" (UniqueName: \"kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.647322 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.647033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:36:39.655941 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.655916    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:36:39.660310 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.660291    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:36:39.662305 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.662278    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90889eb1_58bd_4e0a_802c_3156bbe5ef2f.slice/crio-5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa WatchSource:0}: Error finding container 5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa: Status 404 returned error can't find the container with id 5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa
Oct 30 18:36:39.665137 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.665102    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:36:39.670745 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.670609    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod748b974a_4a23_4a7a_949b_3dc6f972ea61.slice/crio-4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8 WatchSource:0}: Error finding container 4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8: Status 404 returned error can't find the container with id 4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8
Oct 30 18:36:39.681289 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.681266    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l9qs4"
Oct 30 18:36:39.684302 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.683413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:39.684551 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.684532    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279b261c_91e6_427e_9e19_3f1a42e60e76.slice/crio-72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb WatchSource:0}: Error finding container 72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb: Status 404 returned error can't find the container with id 72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb
Oct 30 18:36:39.687131 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.687088    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:36:39.689605 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.689589    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j4266"
Oct 30 18:36:39.702644 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.702585    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf3e1e65_7ce3_4b20_9923_98c273dd915f.slice/crio-9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8 WatchSource:0}: Error finding container 9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8: Status 404 returned error can't find the container with id 9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8
Oct 30 18:36:39.706549 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:39.706532    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:39.708572 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.708546    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca72a21_1b7a_4005_9baa_970f86ebd652.slice/crio-684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073 WatchSource:0}: Error finding container 684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073: Status 404 returned error can't find the container with id 684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073
Oct 30 18:36:39.718075 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:36:39.718049    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861ecda3_10bb_451b_bf57_f62ef9cd4129.slice/crio-5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1 WatchSource:0}: Error finding container 5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1: Status 404 returned error can't find the container with id 5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1
Oct 30 18:36:40.015184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.014870    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:40.015184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.014927    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:40.015184 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.015031    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:40.015184 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.015080    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:36:41.015066177 +0000 UTC m=+3.107902100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:40.015526 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.015442    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:40.015526 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.015505    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:36:41.015478847 +0000 UTC m=+3.108314761 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:40.116054 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.116013    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:40.116231 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.116214    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:40.116317 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.116235    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:40.116317 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.116263    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:40.116423 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.116320    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:41.116302066 +0000 UTC m=+3.209137979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:40.368449 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.368346    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:39 +0000 UTC" deadline="2028-04-09 01:17:02.681159969 +0000 UTC"
Oct 30 18:36:40.368449 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.368402    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12630h40m22.312762168s"
Oct 30 18:36:40.370091 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.369510    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:40.370091 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:40.369685    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:40.392472 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.392442    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l9qs4" event={"ID":"cca6e1a6-ea37-482c-9c61-1a7ac0259030","Type":"ContainerStarted","Data":"f89ad751c0a5eec6f927bbae97c6888f6fcba86508d4526971315bdb0ec62bc5"}
Oct 30 18:36:40.405951 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.405883    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ghjt5" event={"ID":"279b261c-91e6-427e-9e19-3f1a42e60e76","Type":"ContainerStarted","Data":"72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb"}
Oct 30 18:36:40.411646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.411593    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rwlfm" event={"ID":"7308f12c-06da-41a7-b2b1-ab82200985ee","Type":"ContainerStarted","Data":"8249c4a584a8a03ccdaa41c3bcfc08001029c7e848676aa2af2dd19a289afcc5"}
Oct 30 18:36:40.416393 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.416368    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2" event={"ID":"748b974a-4a23-4a7a-949b-3dc6f972ea61","Type":"ContainerStarted","Data":"4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8"}
Oct 30 18:36:40.425593 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.425571    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8tlpv" event={"ID":"90889eb1-58bd-4e0a-802c-3156bbe5ef2f","Type":"ContainerStarted","Data":"5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa"}
Oct 30 18:36:40.435366 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.435342    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1"}
Oct 30 18:36:40.441134 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.441108    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4266" event={"ID":"7ba4b2b7-3196-4769-aaa9-8c40dd282893","Type":"ContainerStarted","Data":"ad1dd7893496396ea9e505450a2a22e9c4869b88153f21bef4f1b0935b4c6b2b"}
Oct 30 18:36:40.444501 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.444476    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerStarted","Data":"9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8"}
Oct 30 18:36:40.446385 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.446365    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bz4h" event={"ID":"fca72a21-1b7a-4005-9baa-970f86ebd652","Type":"ContainerStarted","Data":"684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073"}
Oct 30 18:36:40.728716 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.728579    2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:40.815100 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:40.815069    2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:36:41.023895 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.023809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:41.023895 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.023880    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:41.024107 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.023996    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:41.024107 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.024001    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:41.024107 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.024051    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:36:43.024033941 +0000 UTC m=+5.116869865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:41.024107 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.024068    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:36:43.024059454 +0000 UTC m=+5.116895367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:41.125227 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.125190    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:41.125525 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.125435    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:41.125525 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.125457    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:41.125525 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.125469    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:41.125689 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.125529    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:43.1255111 +0000 UTC m=+5.218347027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:41.367758 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.367687    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:41.367912 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.367834    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:41.368291 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.368270    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:41.368404 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:41.368384    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:41.368631 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.368515    2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:39 +0000 UTC" deadline="2028-05-28 14:33:17.812805314 +0000 UTC"
Oct 30 18:36:41.368631 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:41.368573    2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13819h56m36.444236729s"
Oct 30 18:36:42.368457 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:42.368428    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:42.368596 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:42.368560    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:42.462398 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:42.462044    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"d35ca846eedff7c0a375d6f9a1bb598c","Type":"ContainerStarted","Data":"6f2c5d56636029043503c347efa50791e7dfe02bf598b945e09b2b7b552b66bb"}
Oct 30 18:36:43.042166 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.042078    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:43.042166 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.042156    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:43.042401 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.042295    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:43.042401 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.042355    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.042335348 +0000 UTC m=+9.135171257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:43.042716 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.042699    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:43.042779 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.042751    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.042735234 +0000 UTC m=+9.135571158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:43.143366 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.143324    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:43.143533 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.143480    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:43.143533 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.143500    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:43.143533 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.143512    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:43.143667 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.143562    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:47.143545713 +0000 UTC m=+9.236381639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:43.367644 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.367575    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:43.367794 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.367709    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:43.368102 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.368082    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:43.368200 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:43.368181    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:43.468639 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.468603    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="e5f3a5b8889ec77294e20b56374af895423fed117de18dba110dde8695aafe36" exitCode=0
Oct 30 18:36:43.469091 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.468690    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"e5f3a5b8889ec77294e20b56374af895423fed117de18dba110dde8695aafe36"}
Oct 30 18:36:43.477807 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.477783    2581 generic.go:356] "Generic (PLEG): container finished" podID="22ce239b215936dbe4b6418b6fb4e9c3" containerID="3d9d04d42b9b1ecdbf0b28296bc4394fbc47b4e4e1e3e91736123d443be6918b" exitCode=0
Oct 30 18:36:43.478532 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.478504    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerDied","Data":"3d9d04d42b9b1ecdbf0b28296bc4394fbc47b4e4e1e3e91736123d443be6918b"}
Oct 30 18:36:43.556477 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:43.556392    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" podStartSLOduration=5.55637299 podStartE2EDuration="5.55637299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:42.50315878 +0000 UTC m=+4.595994714" watchObservedRunningTime="2026-10-30 18:36:43.55637299 +0000 UTC m=+5.649208922"
Oct 30 18:36:44.370626 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:44.370551    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:44.370795 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:44.370685    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:44.486347 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:44.485909    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerStarted","Data":"72542513d6f66e7885c9b3827c307c665b2baf03311a5cdd48f0e7cddc5ed536"}
Oct 30 18:36:44.490935 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:44.490502    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6bz4h" event={"ID":"fca72a21-1b7a-4005-9baa-970f86ebd652","Type":"ContainerStarted","Data":"c479881fb511c6beb826ea86b92ef3447c1cd6b569ee8ee5393684567552ee4d"}
Oct 30 18:36:44.568789 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:44.568735    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6bz4h" podStartSLOduration=2.5314912080000003 podStartE2EDuration="6.568722284s" totalImagesPullingTime="4.037231076s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:44.568554887 +0000 UTC m=+6.661390844" watchObservedRunningTime="2026-10-30 18:36:44.568722284 +0000 UTC m=+6.661558214"
Oct 30 18:36:44.569043 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:44.568984    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" podStartSLOduration=6.568977108 podStartE2EDuration="6.568977108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:44.526426043 +0000 UTC m=+6.619261977" watchObservedRunningTime="2026-10-30 18:36:44.568977108 +0000 UTC m=+6.661813041"
Oct 30 18:36:45.368151 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:45.368115    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:45.368361 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:45.368284    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:45.368426 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:45.368365    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:45.368480 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:45.368458    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:45.496712 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:45.496664    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ghjt5" event={"ID":"279b261c-91e6-427e-9e19-3f1a42e60e76","Type":"ContainerStarted","Data":"c02d6de06f857ba0bf65c3ce9b8036374465667522e6f2fc498faf49379dbbc2"}
Oct 30 18:36:46.299644 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:46.299528    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:46.370863 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:46.370352    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:46.370863 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:46.370477    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:46.503158 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:46.502772    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2" event={"ID":"748b974a-4a23-4a7a-949b-3dc6f972ea61","Type":"ContainerStarted","Data":"f44427ccc23093c10bae2da1051c60a4fe9040968d889520fad7f43be66ad70b"}
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.073067    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.073137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.073272    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.073328    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:36:55.073311274 +0000 UTC m=+17.166147206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.073732    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:47.073832 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.073780    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:36:55.073765385 +0000 UTC m=+17.166601300 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:47.174235 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.174170    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:47.174383 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.174351    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:47.174383 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.174371    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:47.174498 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.174386    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:47.174498 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.174438    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:36:55.174419905 +0000 UTC m=+17.267255828 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:47.281905 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.281638    2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:36:47.370156 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.367508    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:47.370156 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.367563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:47.370156 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.367704    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:47.370156 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:47.368198    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:47.373060 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.372324    2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:36:47.281657682Z","UUID":"d9f7c3c6-07fd-4b94-87ee-1c4d07d7ff06","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:36:47.374288 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.374261    2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:36:47.374288 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.374289    2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:36:47.508871 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.508767    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2" event={"ID":"748b974a-4a23-4a7a-949b-3dc6f972ea61","Type":"ContainerStarted","Data":"7d16ebf83e4d06a1681f5de0b429aa6949d03d0d6677681ec11b1b6e102b2dc1"}
Oct 30 18:36:47.945892 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.945777    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:47.946872 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.946849    2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:47.986180 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:47.986137    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ghjt5" podStartSLOduration=4.86270517 podStartE2EDuration="9.986124519s" totalImagesPullingTime="5.123419349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:45.536027467 +0000 UTC m=+7.628863399" watchObservedRunningTime="2026-10-30 18:36:47.986124519 +0000 UTC m=+10.078960449"
Oct 30 18:36:48.368851 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.368825    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:48.368984 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:48.368898    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:48.512635 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.512545    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j4266" event={"ID":"7ba4b2b7-3196-4769-aaa9-8c40dd282893","Type":"ContainerStarted","Data":"cd8ce72d67746132e56f4b70cd31069a29038cff18ee7508755462ebe6f2da00"}
Oct 30 18:36:48.514735 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.514701    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="558de952087e59ba4596078e0970b0fd937b4f5dc58ed1fb7a1f739b4f7b2b23" exitCode=0
Oct 30 18:36:48.514855 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.514795    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"558de952087e59ba4596078e0970b0fd937b4f5dc58ed1fb7a1f739b4f7b2b23"}
Oct 30 18:36:48.516814 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.516783    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l9qs4" event={"ID":"cca6e1a6-ea37-482c-9c61-1a7ac0259030","Type":"ContainerStarted","Data":"578bb7c4f65933e807bc929dd9002ce8b546cdc4a4cf1bdbb76294e8c6c7497f"}
Oct 30 18:36:48.520331 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.520304    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2" event={"ID":"748b974a-4a23-4a7a-949b-3dc6f972ea61","Type":"ContainerStarted","Data":"12ac93796c49824ee49912a164e242f745d8784788797de4135dac5c2e50f79b"}
Oct 30 18:36:48.520814 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.520794    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:36:48.593008 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.592777    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-j4266" podStartSLOduration=2.440571544 podStartE2EDuration="10.592763759s" totalImagesPullingTime="8.152192215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:48.552339267 +0000 UTC m=+10.645175199" watchObservedRunningTime="2026-10-30 18:36:48.592763759 +0000 UTC m=+10.685599690"
Oct 30 18:36:48.593417 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.593378    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2" podStartSLOduration=2.397239214 podStartE2EDuration="10.593366979s" totalImagesPullingTime="8.196127765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:48.592404376 +0000 UTC m=+10.685240307" watchObservedRunningTime="2026-10-30 18:36:48.593366979 +0000 UTC m=+10.686202910"
Oct 30 18:36:48.721639 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:48.721590    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-l9qs4" podStartSLOduration=2.991574202 podStartE2EDuration="10.721572293s" totalImagesPullingTime="7.729998091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:48.6836869 +0000 UTC m=+10.776522832" watchObservedRunningTime="2026-10-30 18:36:48.721572293 +0000 UTC m=+10.814408223"
Oct 30 18:36:49.367919 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.367840    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:49.367919 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.367861    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:49.368110 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:49.368050    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:49.368205 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:49.368181    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:49.524067 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.524026    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8tlpv" event={"ID":"90889eb1-58bd-4e0a-802c-3156bbe5ef2f","Type":"ContainerStarted","Data":"5202b4a93a77e6fa10518d356ef2ac89becac179a8a1238419ca81d201e55ef6"}
Oct 30 18:36:49.526705 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.526666    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="afa2a2535f2483c7272d1d7a550d7000aa7a9cc8d74aa497cf29878a1fb0e9a2" exitCode=0
Oct 30 18:36:49.527588 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.527566    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"afa2a2535f2483c7272d1d7a550d7000aa7a9cc8d74aa497cf29878a1fb0e9a2"}
Oct 30 18:36:49.607140 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:49.607087    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8tlpv" podStartSLOduration=3.679264984 podStartE2EDuration="11.607073527s" totalImagesPullingTime="7.927808543s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:49.562611692 +0000 UTC m=+11.655447622" watchObservedRunningTime="2026-10-30 18:36:49.607073527 +0000 UTC m=+11.699909458"
Oct 30 18:36:50.367511 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:50.367487    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:50.367620 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:50.367601    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:50.530235 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:50.530185    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rwlfm" event={"ID":"7308f12c-06da-41a7-b2b1-ab82200985ee","Type":"ContainerStarted","Data":"8dee47912ec57397ff108c83950c54afbb26cc36b46c0f9e907111272c6b5c0a"}
Oct 30 18:36:50.536447 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:50.536413    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="86efa1157f5a9e3a8e1fb04c3ac7bdab11434ce0d575cd0351d3f7bc977716cd" exitCode=0
Oct 30 18:36:50.536587 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:50.536507    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"86efa1157f5a9e3a8e1fb04c3ac7bdab11434ce0d575cd0351d3f7bc977716cd"}
Oct 30 18:36:50.572036 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:50.571909    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rwlfm" podStartSLOduration=2.188731991 podStartE2EDuration="12.571892948s" totalImagesPullingTime="10.383160957s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:50.571454849 +0000 UTC m=+12.664290798" watchObservedRunningTime="2026-10-30 18:36:50.571892948 +0000 UTC m=+12.664728880"
Oct 30 18:36:51.368422 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:51.368389    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:51.368587 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:51.368438    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:51.368587 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:51.368508    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:51.368691 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:51.368644    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:52.368472 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:52.368440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:52.368995 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:52.368560    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:52.544403 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:52.544315    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="0a99cd508fb248a31b669c8713b78e6c5cf0792331f263b64c2d013656856d94" exitCode=0
Oct 30 18:36:52.544403 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:52.544370    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"0a99cd508fb248a31b669c8713b78e6c5cf0792331f263b64c2d013656856d94"}
Oct 30 18:36:53.368602 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:53.368359    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:53.368992 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:53.368398    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:53.368992 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:53.368728    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:53.368992 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:53.368797    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:53.550573 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:53.550540    2581 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="943d35aaa9714a5961e2e16712daa84d8f7be8abd8801e93263fcd5b9c17f8c4" exitCode=0
Oct 30 18:36:53.550573 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:53.550584    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerDied","Data":"943d35aaa9714a5961e2e16712daa84d8f7be8abd8801e93263fcd5b9c17f8c4"}
Oct 30 18:36:54.367813 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.367786    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:54.367949 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:54.367923    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:54.556596 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.556364    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:36:54.557010 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.556986    2581 generic.go:356] "Generic (PLEG): container finished" podID="861ecda3-10bb-451b-bf57-f62ef9cd4129" containerID="5bcafd92b13acaa4c5a973b2648903b52b2e79a8d903a4a8540aaf7f0bbb73da" exitCode=1
Oct 30 18:36:54.557052 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557022    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"bc77d69759b9a08b238dac526503eb2c39c5e575b379f968a5e7798b2fc42967"}
Oct 30 18:36:54.557127 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557113    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"c6a9db871490f796b6b57a88b2fd3827048bf83331780247366c7468ecf5673e"}
Oct 30 18:36:54.557177 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557154    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"98847f565f91ff58e4886cbe7e8360df7aa8ccf4c17ceefdb1a1ac019f226110"}
Oct 30 18:36:54.557222 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557197    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"f8344b37d3e1e3d0fd63d0b804600dde1a1db11376eb25aac0917ce35828b378"}
Oct 30 18:36:54.557277 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557229    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerDied","Data":"5bcafd92b13acaa4c5a973b2648903b52b2e79a8d903a4a8540aaf7f0bbb73da"}
Oct 30 18:36:54.557314 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.557282    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"01dd7af388d0c98f7b0f2e2ce4defe6429440d8cf2ceae25f4670f4d5c794ce0"}
Oct 30 18:36:54.560494 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.560472    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" event={"ID":"cf3e1e65-7ce3-4b20-9923-98c273dd915f","Type":"ContainerStarted","Data":"1f60a0a29dfe5d239d1715b0223637d8ac9b3cad2a7b3cde7d395b414407b4be"}
Oct 30 18:36:54.606321 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:54.606233    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-n9ff9" podStartSLOduration=6.35580257 podStartE2EDuration="16.606218584s" totalImagesPullingTime="10.250416014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:36:54.605884207 +0000 UTC m=+16.698720117" watchObservedRunningTime="2026-10-30 18:36:54.606218584 +0000 UTC m=+16.699054516"
Oct 30 18:36:55.139136 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:55.139108    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:55.139301 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:55.139157    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:55.139301 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.139285    2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:55.139366 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.139340    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret podName:0e6538b1-d3ff-4e67-81e0-e147ce5d03db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.139327389 +0000 UTC m=+33.232163298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret") pod "global-pull-secret-syncer-kp5w5" (UID: "0e6538b1-d3ff-4e67-81e0-e147ce5d03db") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:36:55.139410 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.139285    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:55.139410 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.139408    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.139393452 +0000 UTC m=+33.232229374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:36:55.239771 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:55.239741    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:55.239902 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.239887    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:36:55.239937 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.239907    2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:36:55.239937 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.239917    2581 projected.go:196] Error preparing data for projected volume kube-api-access-swtrr for pod openshift-network-diagnostics/network-check-target-kps9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:55.240008 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.239959    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr podName:16f258b8-0324-4e3a-85a1-50c4194f593d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.239946888 +0000 UTC m=+33.332782798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-swtrr" (UniqueName: "kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr") pod "network-check-target-kps9t" (UID: "16f258b8-0324-4e3a-85a1-50c4194f593d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:36:55.367932 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:55.367898    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:55.368066 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:55.367933    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:55.368066 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.367997    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:55.368134 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:55.368122    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:56.367950 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:56.367915    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:56.368354 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:56.368016    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:56.567645 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:56.567620    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:36:57.368105 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:57.368071    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:57.368105 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:57.368090    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:57.368645 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:57.368195    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:57.368645 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:57.368333    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:57.572835 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:57.572808    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:36:57.573165 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:57.573144    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"8c71ae750d304f4a7ea9fea02b67809e3aa35ebcbcdad781d3e85053a7ecf2ad"}
Oct 30 18:36:58.368259 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:58.368231    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:36:58.368760 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:58.368337    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:36:59.367891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.367866    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:36:59.368040 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:59.367964    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:36:59.368040 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.368022    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:36:59.368226 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:36:59.368115    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:36:59.580085 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.580059    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:36:59.580898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.580389    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"9b92e014125e94653594abc942fee93b2a0ff59abe2aad7e8eb82540033a2a49"}
Oct 30 18:36:59.580898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.580680    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:59.580898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.580800    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:59.580898 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.580852    2581 scope.go:122] "RemoveContainer" containerID="5bcafd92b13acaa4c5a973b2648903b52b2e79a8d903a4a8540aaf7f0bbb73da"
Oct 30 18:36:59.597031 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.597008    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:36:59.597133 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:36:59.597105    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:00.368444 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.368413    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:00.368617 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:00.368548    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:37:00.587624 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.587393    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:37:00.588109 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.588081    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" event={"ID":"861ecda3-10bb-451b-bf57-f62ef9cd4129","Type":"ContainerStarted","Data":"4d98866418aa15714e1088df7e168eee21492f8444f35e32a2d524d9d98eb0f7"}
Oct 30 18:37:00.588199 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.588182    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:37:00.648293 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.648187    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz" podStartSLOduration=8.289309023 podStartE2EDuration="22.648167412s" totalImagesPullingTime="14.358858389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:00.647410951 +0000 UTC m=+22.740246902" watchObservedRunningTime="2026-10-30 18:37:00.648167412 +0000 UTC m=+22.741003344"
Oct 30 18:37:00.661528 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.661502    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlgx9"]
Oct 30 18:37:00.661665 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.661604    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:00.661724 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:00.661702    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:37:00.662106 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.662089    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kps9t"]
Oct 30 18:37:00.662175 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.662160    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:00.662260 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:00.662233    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:37:00.662637 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.662618    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kp5w5"]
Oct 30 18:37:00.662720 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:00.662707    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:00.662821 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:00.662802    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:37:01.590031 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:01.589937    2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:37:01.871832 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:01.871764    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:02.367557 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:02.367475    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:02.367760 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:02.367490    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:02.367760 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:02.367579    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:37:02.367760 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:02.367596    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:02.367760 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:02.367672    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:37:02.367760 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:02.367719    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:37:04.367737 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:04.367705    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:04.368116 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:04.367706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:04.368116 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:04.367814    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:37:04.368116 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:04.367706    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:04.368116 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:04.367877    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:37:04.368116 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:04.367938    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:37:06.367649 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:06.367623    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:06.368083 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:06.367657    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:06.368083 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:06.367715    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kp5w5" podUID="0e6538b1-d3ff-4e67-81e0-e147ce5d03db"
Oct 30 18:37:06.368083 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:06.367757    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:06.368083 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:06.367858    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlgx9" podUID="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc"
Oct 30 18:37:06.368083 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:06.367928    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kps9t" podUID="16f258b8-0324-4e3a-85a1-50c4194f593d"
Oct 30 18:37:07.169077 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.169018    2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeReady"
Oct 30 18:37:07.169206 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.169138    2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:37:07.216112 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.216082    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-96hgm"]
Oct 30 18:37:07.219934 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.219903    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:07.220840 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.220818    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zh56w"]
Oct 30 18:37:07.224906 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.224887    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5kbfh"]
Oct 30 18:37:07.225028 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.225009    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:07.228287 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.228268    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"]
Oct 30 18:37:07.228364 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.228316    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:07.232044 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.231307    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"]
Oct 30 18:37:07.232411 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.232391    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.234915 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.234174    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"]
Oct 30 18:37:07.234915 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.234303    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.238358 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.238341    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-rzcch"]
Oct 30 18:37:07.238477 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.238461    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.242334 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.242317    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c6947b789-btt56"]
Oct 30 18:37:07.242455 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.242440    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.243839 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.243822    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.243908 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.243829    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.245291 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.245241    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"]
Oct 30 18:37:07.245385 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.245324    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.253127 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.253082    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:37:07.253220 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.253198    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.261040    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7rwzw\""
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.261296    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.261364    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.262129    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.262236    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nczkq\""
Oct 30 18:37:07.263891 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.262352    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-g4ls8\""
Oct 30 18:37:07.264267 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.264218    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-mvhgb"]
Oct 30 18:37:07.264550 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.264529    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.265603 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.265586    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dvz9x\""
Oct 30 18:37:07.268892 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.268873    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"]
Oct 30 18:37:07.269718 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.269700    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.273720 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.273701    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"]
Oct 30 18:37:07.274394 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.274375    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.276099 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.276081    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:07.276221 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.276203    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:07.279602 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.279582    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"]
Oct 30 18:37:07.280069 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.280046    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.282692 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.282676    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6w2g6"]
Oct 30 18:37:07.282793 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.282780    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:07.286040 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.286022    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tkz7q"]
Oct 30 18:37:07.286143 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.286127    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.290017 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290000    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-96hgm"]
Oct 30 18:37:07.290175 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290161    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zh56w"]
Oct 30 18:37:07.290338 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290301    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5kbfh"]
Oct 30 18:37:07.290338 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290334    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"]
Oct 30 18:37:07.290468 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290356    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"]
Oct 30 18:37:07.290468 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290376    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"]
Oct 30 18:37:07.290468 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290394    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290493    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c6947b789-btt56"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290519    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290025    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290547    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-mvhgb"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290572    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290590    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"]
Oct 30 18:37:07.290671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.290617    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-rzcch"]
Oct 30 18:37:07.292367 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.292351    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tkz7q"]
Oct 30 18:37:07.293200 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.293179    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6w2g6"]
Oct 30 18:37:07.296048 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.295892    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:07.296818 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.296800    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:37:07.327541 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327518    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.327716 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327693    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.327819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327724    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fhfx\" (UniqueName: \"kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.327819 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327771    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.327938 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327832    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.327938 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327872    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.327938 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327902    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff5qc\" (UniqueName: \"kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc\") pod \"kas-connection-checker-685895c7c9-zh56w\" (UID: \"7265c58c-0d6e-4fe4-98af-9db6238dd3b8\") " pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327951    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.327978    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328009    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328029    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328045    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wrq2\" (UniqueName: \"kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2\") pod \"kas-connection-checker-685895c7c9-5kbfh\" (UID: \"e69d8d91-a4a3-412d-ab1b-d9a8409580fa\") " pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328059    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328073    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328090    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnl2m\" (UniqueName: \"kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328119    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328145    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfppz\" (UniqueName: \"kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.328194 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328182    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328213    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n7l9\" (UniqueName: \"kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9\") pod \"kas-connection-checker-685895c7c9-96hgm\" (UID: \"4d127029-258f-4c97-a9b8-d2cf85123890\") " pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328241    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328290    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328317    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328347    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328379    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k87cx\" (UniqueName: \"kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328548    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328581    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328624    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328651    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328704    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqnln\" (UniqueName: \"kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328723    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328754    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmgkh\" (UniqueName: \"kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.328780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.328773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.340931 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.340888    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.341111 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.341090    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:37:07.341337 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.341318    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.341420 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.341029    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:37:07.341472 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.340998    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.341780 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.341108    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:37:07.344136 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344040    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.344229 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344214    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.344323 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344309    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:37:07.344433 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344388    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.344484 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344437    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.344529 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344492    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.344694 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344678    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.344752 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344741    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.344833 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344820    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:37:07.344887 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344876    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.344951 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344940    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.344999 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.344986    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:37:07.345044 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.345038    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.345172 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.345157    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.345221 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.345179    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:37:07.345614 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.345596    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.345882 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.345867    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.346056 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.346044    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.346190 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.346169    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.346235 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.346171    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:07.346235 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.346206    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:37:07.346235 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.346207    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:07.355746 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.355704    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:37:07.356612 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.356590    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:37:07.356708 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.356675    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:37:07.356708 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.356693    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:37:07.360941 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.360922    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:37:07.361092 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361071    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:37:07.361184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361126    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:37:07.361184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361134    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:37:07.361184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361140    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:37:07.361184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361154    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bx2pm\""
Oct 30 18:37:07.361184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361154    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b8l82\""
Oct 30 18:37:07.361688 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361236    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vjc5x\""
Oct 30 18:37:07.361688 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.361360    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:37:07.367956 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.367938    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:37:07.368302 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.367944    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:37:07.368356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.367965    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:37:07.368356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.367977    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:37:07.368356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368007    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mdkqz\""
Oct 30 18:37:07.368356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368029    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:37:07.368356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368077    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t7qvq\""
Oct 30 18:37:07.368578 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368501    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:37:07.368578 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368550    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-64xr6\""
Oct 30 18:37:07.368578 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368554    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gtrxb\""
Oct 30 18:37:07.368578 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368573    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mxkzp\""
Oct 30 18:37:07.368693 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.368606    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6dtmn\""
Oct 30 18:37:07.429671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429617    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.429671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429644    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ng5d\" (UniqueName: \"kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.429671 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429663    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.429837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429685    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.429837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429739    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.429837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429809    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429845    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fhfx\" (UniqueName: \"kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429871    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7jhr\" (UniqueName: \"kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.429879    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.429888    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.429892    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bbf88fc59-82jt6: secret "image-registry-tls" not found
Oct 30 18:37:07.429986 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.429943    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls podName:50f0c5e2-b263-4047-b980-20b8ebb07a79 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:07.929929717 +0000 UTC m=+30.022765625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls") pod "image-registry-5bbf88fc59-82jt6" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79") : secret "image-registry-tls" not found
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430065    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430085    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430103    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430121    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430137    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff5qc\" (UniqueName: \"kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc\") pod \"kas-connection-checker-685895c7c9-zh56w\" (UID: \"7265c58c-0d6e-4fe4-98af-9db6238dd3b8\") " pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:07.430234 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430157    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430235    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430292    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.430306    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430359    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.430375    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:07.930357338 +0000 UTC m=+30.023193260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.430422    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.430461    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls podName:f46a5e6f-f333-4f27-a660-2cb0bf7c3470 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:07.930449549 +0000 UTC m=+30.023285462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-klrgt" (UID: "f46a5e6f-f333-4f27-a660-2cb0bf7c3470") : secret "samples-operator-tls" not found
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430483    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.430523 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430507    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430557    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wrq2\" (UniqueName: \"kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2\") pod \"kas-connection-checker-685895c7c9-5kbfh\" (UID: \"e69d8d91-a4a3-412d-ab1b-d9a8409580fa\") " pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430583    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430697    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430731    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnl2m\" (UniqueName: \"kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430774    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430801    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430837    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfppz\" (UniqueName: \"kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430897    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430924    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.430981 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430962    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n7l9\" (UniqueName: \"kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9\") pod \"kas-connection-checker-685895c7c9-96hgm\" (UID: \"4d127029-258f-4c97-a9b8-d2cf85123890\") " pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.430992    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431017    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431021    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431071    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431327    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431339    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431360    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431385    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431409    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.431459 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431441    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431467    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wwgx\" (UniqueName: \"kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431496    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx997\" (UniqueName: \"kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997\") pod \"volume-data-source-validator-58c9f5f46d-q4xsh\" (UID: \"310d68ba-e235-4caf-ba78-c1940e74354c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431533    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87cx\" (UniqueName: \"kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431592    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431619    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431692    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431721    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l96t\" (UniqueName: \"kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t\") pod \"network-check-source-6f78d59f99-49spc\" (UID: \"de24546a-df74-492e-98f2-6dcf0d466108\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431748    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431773    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431795    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431830    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqnln\" (UniqueName: \"kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.431849 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431854    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431877    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431907    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgkh\" (UniqueName: \"kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.431932    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.432032    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.432276    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.432492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.432345    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.432840 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.432797    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.432920 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.432895    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.432977 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.432931    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:07.932914401 +0000 UTC m=+30.025750332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:07.433528 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.433506    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:37:07.433620 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.433579    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:07.9335619 +0000 UTC m=+30.026397823 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : secret "router-metrics-certs-default" not found
Oct 30 18:37:07.434421 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.434401    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.435010 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.434984    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.435110 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.434988    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.435110 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.435080    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.435379 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.435361    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.435379 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.435371    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.435843 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.435809    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.436480 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.436460    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.451822 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.451800    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:07.454162 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.454134    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff5qc\" (UniqueName: \"kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc\") pod \"kas-connection-checker-685895c7c9-zh56w\" (UID: \"7265c58c-0d6e-4fe4-98af-9db6238dd3b8\") " pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:07.454352 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.454328    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fhfx\" (UniqueName: \"kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.454450 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.454427    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.457113 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.457088    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqnln\" (UniqueName: \"kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.457219 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.457179    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgkh\" (UniqueName: \"kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.457727 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.457708    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfppz\" (UniqueName: \"kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.457905 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.457884    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnl2m\" (UniqueName: \"kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.458458 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.458437    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n7l9\" (UniqueName: \"kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9\") pod \"kas-connection-checker-685895c7c9-96hgm\" (UID: \"4d127029-258f-4c97-a9b8-d2cf85123890\") " pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:07.458657 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.458637    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.459629 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.459613    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wrq2\" (UniqueName: \"kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2\") pod \"kas-connection-checker-685895c7c9-5kbfh\" (UID: \"e69d8d91-a4a3-412d-ab1b-d9a8409580fa\") " pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:07.459837 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.459817    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87cx\" (UniqueName: \"kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.532259 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532232    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.532377 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532280    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532377 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532313    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.532377 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532339    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532377 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532370    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532400    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532422    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532446    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532470    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532494    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532524    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wwgx\" (UniqueName: \"kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.532571 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532550    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx997\" (UniqueName: \"kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997\") pod \"volume-data-source-validator-58c9f5f46d-q4xsh\" (UID: \"310d68ba-e235-4caf-ba78-c1940e74354c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532610    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532633    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l96t\" (UniqueName: \"kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t\") pod \"network-check-source-6f78d59f99-49spc\" (UID: \"de24546a-df74-492e-98f2-6dcf0d466108\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532660    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532683    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532727    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532761    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.532446    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:07.532901 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.532859    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.032844748 +0000 UTC m=+30.125680671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:07.533356 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.532765    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.533740 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.533709    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.534188 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534164    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.534357 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534339    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534528    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ng5d\" (UniqueName: \"kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534577    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534609    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.532497    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.534706    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8457959978-p9454: secret "image-registry-tls" not found
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.534775    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls podName:82214171-38b1-45d5-8c9b-794a142335fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.034752024 +0000 UTC m=+30.127587954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls") pod "image-registry-8457959978-p9454" (UID: "82214171-38b1-45d5-8c9b-794a142335fc") : secret "image-registry-tls" not found
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535124    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535235    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.534673    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jhr\" (UniqueName: \"kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535336    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.535499 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535378    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.536023 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535534    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.536023 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.535557    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:07.536023 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.535619    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.035594182 +0000 UTC m=+30.128430105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:07.536023 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535705    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:07.536023 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.535979    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.536242 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.536108    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:07.536515 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.536484    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.537745 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.537687    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.537838 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.537751    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.540449 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.540426    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.541491 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.541470    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.542028 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.542008    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:07.552147 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.552129    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:07.556447 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.556421    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ng5d\" (UniqueName: \"kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:07.556643 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.556622    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wwgx\" (UniqueName: \"kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.556822 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.556804    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jhr\" (UniqueName: \"kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.557554 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.557272    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx997\" (UniqueName: \"kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997\") pod \"volume-data-source-validator-58c9f5f46d-q4xsh\" (UID: \"310d68ba-e235-4caf-ba78-c1940e74354c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:07.558154 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.558130    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.558564 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.558541    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:07.558836 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.558813    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l96t\" (UniqueName: \"kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t\") pod \"network-check-source-6f78d59f99-49spc\" (UID: \"de24546a-df74-492e-98f2-6dcf0d466108\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:07.571709 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.571677    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:07.579229 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.579199    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:07.587482 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.587383    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:07.622836 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.622790    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:07.637089 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.637060    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:07.654634 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.654600    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:07.659743 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.659663    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:07.671050 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.670982    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:07.902297 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.902277    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-zh56w"]
Oct 30 18:37:07.904945 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:07.904918    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7265c58c_0d6e_4fe4_98af_9db6238dd3b8.slice/crio-19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c WatchSource:0}: Error finding container 19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c: Status 404 returned error can't find the container with id 19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c
Oct 30 18:37:07.905954 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.905933    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-96hgm"]
Oct 30 18:37:07.910881 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:07.910839    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d127029_258f_4c97_a9b8_d2cf85123890.slice/crio-154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d WatchSource:0}: Error finding container 154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d: Status 404 returned error can't find the container with id 154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d
Oct 30 18:37:07.912645 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.912488    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"]
Oct 30 18:37:07.915318 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.915291    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-5kbfh"]
Oct 30 18:37:07.920504 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.920398    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"]
Oct 30 18:37:07.923678 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:07.923658    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77ce2b35_2046_4b1a_bb36_7f49fe33f506.slice/crio-cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a WatchSource:0}: Error finding container cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a: Status 404 returned error can't find the container with id cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a
Oct 30 18:37:07.941031 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.940996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.941128 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.941070    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:07.941128 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.941112    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:07.941239 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.941168    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:07.941239 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.941219    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:07.941469 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941408    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:37:07.941469 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941467    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls podName:f46a5e6f-f333-4f27-a660-2cb0bf7c3470 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.941448428 +0000 UTC m=+31.034284356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-klrgt" (UID: "f46a5e6f-f333-4f27-a660-2cb0bf7c3470") : secret "samples-operator-tls" not found
Oct 30 18:37:07.941617 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941546    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.941534669 +0000 UTC m=+31.034370594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:07.941617 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941607    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:37:07.941721 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941638    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.941627722 +0000 UTC m=+31.034463648 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : secret "router-metrics-certs-default" not found
Oct 30 18:37:07.941721 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941701    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:07.941721 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941713    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bbf88fc59-82jt6: secret "image-registry-tls" not found
Oct 30 18:37:07.941866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941745    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls podName:50f0c5e2-b263-4047-b980-20b8ebb07a79 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.941735448 +0000 UTC m=+31.034571372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls") pod "image-registry-5bbf88fc59-82jt6" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79") : secret "image-registry-tls" not found
Oct 30 18:37:07.941866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941801    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:07.941866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:07.941834    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:08.941824109 +0000 UTC m=+31.034660032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:07.978229 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:07.978201    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-rzcch"]
Oct 30 18:37:08.028199 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.028121    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.041984    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.042069    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042086    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042100    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8457959978-p9454: secret "image-registry-tls" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.042116    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042142    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls podName:82214171-38b1-45d5-8c9b-794a142335fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:09.042128845 +0000 UTC m=+31.134964773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls") pod "image-registry-8457959978-p9454" (UID: "82214171-38b1-45d5-8c9b-794a142335fc") : secret "image-registry-tls" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042192    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042194    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042215    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:09.042208061 +0000 UTC m=+31.135043970 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:08.042241 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.042229    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:09.042219142 +0000 UTC m=+31.135055070 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:08.063659 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.063632    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tkz7q"]
Oct 30 18:37:08.064266 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:08.063824    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde24546a_df74_492e_98f2_6dcf0d466108.slice/crio-218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12 WatchSource:0}: Error finding container 218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12: Status 404 returned error can't find the container with id 218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12
Oct 30 18:37:08.065679 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:08.065655    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310d68ba_e235_4caf_ba78_c1940e74354c.slice/crio-973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280 WatchSource:0}: Error finding container 973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280: Status 404 returned error can't find the container with id 973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280
Oct 30 18:37:08.066512 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:08.065885    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcff1eb5a_0fc0_47ad_adff_12489a13e317.slice/crio-e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1 WatchSource:0}: Error finding container e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1: Status 404 returned error can't find the container with id e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1
Oct 30 18:37:08.067331 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.067310    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"]
Oct 30 18:37:08.069328 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.069308    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"]
Oct 30 18:37:08.070696 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.070675    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-mvhgb"]
Oct 30 18:37:08.178812 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.178783    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:08.369237 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.369159    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:08.369827 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.369351    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:08.369827 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.369764    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:08.385706 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.385680    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9ktdg\""
Oct 30 18:37:08.385894 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.385879    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:37:08.385957 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.385947    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zcgpq\""
Oct 30 18:37:08.386026 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.386008    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:37:08.386076 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.386059    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d48sc\""
Oct 30 18:37:08.614597 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.614226    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" event={"ID":"808b5fa2-8623-4639-870a-be16d38cafeb","Type":"ContainerStarted","Data":"28ed2be66f04084c9eca971198c00b98fd3e89a525b0a5a2643f3660dfa6f62e"}
Oct 30 18:37:08.618175 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.618139    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9" event={"ID":"77ce2b35-2046-4b1a-bb36-7f49fe33f506","Type":"ContainerStarted","Data":"cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a"}
Oct 30 18:37:08.622355 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.621525    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5kbfh" event={"ID":"e69d8d91-a4a3-412d-ab1b-d9a8409580fa","Type":"ContainerStarted","Data":"f83e0dce1e94b2b55e654728a7f98bf32d4f7058d4840213fc76b37cc53f0be3"}
Oct 30 18:37:08.622355 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.621573    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-5kbfh" event={"ID":"e69d8d91-a4a3-412d-ab1b-d9a8409580fa","Type":"ContainerStarted","Data":"3af857443715187d65401c9b3b5941a739ac0b2307510d8b40a1212945eda39e"}
Oct 30 18:37:08.623871 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.623844    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tkz7q" event={"ID":"cff1eb5a-0fc0-47ad-adff-12489a13e317","Type":"ContainerStarted","Data":"e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1"}
Oct 30 18:37:08.625490 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.625465    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc" event={"ID":"de24546a-df74-492e-98f2-6dcf0d466108","Type":"ContainerStarted","Data":"218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12"}
Oct 30 18:37:08.626889 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.626828    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh" event={"ID":"310d68ba-e235-4caf-ba78-c1940e74354c","Type":"ContainerStarted","Data":"973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280"}
Oct 30 18:37:08.629884 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.629859    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8" event={"ID":"e27601fc-9881-482b-9115-a169779efa5b","Type":"ContainerStarted","Data":"a403a38582bc34a4a061361dd926dd85c83ef5e684d6394a57bba53043fd5140"}
Oct 30 18:37:08.633289 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.633265    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-mvhgb" event={"ID":"bf0111a4-adeb-451d-a0c2-17469dce5b90","Type":"ContainerStarted","Data":"a31ee4704c1a7f3a9b874a489ba8eb27cea9c90cc4571ce6caf8ea0a06f1f6d3"}
Oct 30 18:37:08.635588 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.635555    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-zh56w" event={"ID":"7265c58c-0d6e-4fe4-98af-9db6238dd3b8","Type":"ContainerStarted","Data":"ce123ce631fa0b1751ece6b42927a4b0841473341017d85e732624667f19eda5"}
Oct 30 18:37:08.635673 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.635601    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-zh56w" event={"ID":"7265c58c-0d6e-4fe4-98af-9db6238dd3b8","Type":"ContainerStarted","Data":"19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c"}
Oct 30 18:37:08.637972 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.637769    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-96hgm" event={"ID":"4d127029-258f-4c97-a9b8-d2cf85123890","Type":"ContainerStarted","Data":"34279ee57e9cc844002b32ab8c5dd29803ce4c48c2f6a876ab68b64284b2c06b"}
Oct 30 18:37:08.637972 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.637808    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-96hgm" event={"ID":"4d127029-258f-4c97-a9b8-d2cf85123890","Type":"ContainerStarted","Data":"154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d"}
Oct 30 18:37:08.665726 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.665674    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-5kbfh" podStartSLOduration=224.665656813 podStartE2EDuration="3m44.665656813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.664471081 +0000 UTC m=+30.757307013" watchObservedRunningTime="2026-10-30 18:37:08.665656813 +0000 UTC m=+30.758492746"
Oct 30 18:37:08.711874 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.711810    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-96hgm" podStartSLOduration=224.711793237 podStartE2EDuration="3m44.711793237s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.710919578 +0000 UTC m=+30.803755511" watchObservedRunningTime="2026-10-30 18:37:08.711793237 +0000 UTC m=+30.804629170"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.951384    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.951472    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.951517    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.951568    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:08.951641    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.951753    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:10.951740596 +0000 UTC m=+33.044576519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952056    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952095    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:10.952085306 +0000 UTC m=+33.044921230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : secret "router-metrics-certs-default" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952136    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952143    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bbf88fc59-82jt6: secret "image-registry-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952162    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls podName:50f0c5e2-b263-4047-b980-20b8ebb07a79 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:10.952156407 +0000 UTC m=+33.044992329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls") pod "image-registry-5bbf88fc59-82jt6" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79") : secret "image-registry-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952196    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952215    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:10.952209234 +0000 UTC m=+33.045045156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952262    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:37:08.952363 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:08.952282    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls podName:f46a5e6f-f333-4f27-a660-2cb0bf7c3470 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:10.952275701 +0000 UTC m=+33.045111622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-klrgt" (UID: "f46a5e6f-f333-4f27-a660-2cb0bf7c3470") : secret "samples-operator-tls" not found
Oct 30 18:37:09.052492 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:09.052449    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:09.052654 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:09.052545    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:09.052654 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:09.052598    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:09.052800 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.052769    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:09.052800 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.052786    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8457959978-p9454: secret "image-registry-tls" not found
Oct 30 18:37:09.052924 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.052838    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls podName:82214171-38b1-45d5-8c9b-794a142335fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.052821394 +0000 UTC m=+33.145657320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls") pod "image-registry-8457959978-p9454" (UID: "82214171-38b1-45d5-8c9b-794a142335fc") : secret "image-registry-tls" not found
Oct 30 18:37:09.053215 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.053187    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:09.053382 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.053264    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.053233267 +0000 UTC m=+33.146069395 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:09.053382 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.053324    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:09.053382 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:09.053356    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:11.05334592 +0000 UTC m=+33.146181831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:10.650857 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.650770    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh" event={"ID":"310d68ba-e235-4caf-ba78-c1940e74354c","Type":"ContainerStarted","Data":"4c4ed502004a1b921bd78c54acca440bec25d8b7c7664cb8da4a1c744a90afd3"}
Oct 30 18:37:10.695317 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.695266    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh" podStartSLOduration=174.483555207 podStartE2EDuration="2m56.695237158s" totalImagesPullingTime="2.211681951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:10.694588743 +0000 UTC m=+32.787424685" watchObservedRunningTime="2026-10-30 18:37:10.695237158 +0000 UTC m=+32.788073067"
Oct 30 18:37:10.695466 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.695394    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-zh56w" podStartSLOduration=226.695390025 podStartE2EDuration="3m46.695390025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.786873626 +0000 UTC m=+30.879709560" watchObservedRunningTime="2026-10-30 18:37:10.695390025 +0000 UTC m=+32.788225955"
Oct 30 18:37:10.842008 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.841970    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:10.974501 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.973638    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:10.974501 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.973773    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.973751642 +0000 UTC m=+37.066587568 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:10.974501 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.974211    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:10.974501 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.974268    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974526    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974544    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bbf88fc59-82jt6: secret "image-registry-tls" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974584    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974595    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls podName:50f0c5e2-b263-4047-b980-20b8ebb07a79 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.974579266 +0000 UTC m=+37.067415190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls") pod "image-registry-5bbf88fc59-82jt6" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79") : secret "image-registry-tls" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974628    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.97461434 +0000 UTC m=+37.067450249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974673    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:37:10.974756 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974715    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.974706727 +0000 UTC m=+37.067542649 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : secret "router-metrics-certs-default" not found
Oct 30 18:37:10.975084 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.974326    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:10.975084 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.974839    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:10.975084 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.974973    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:37:10.975084 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:10.975036    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls podName:f46a5e6f-f333-4f27-a660-2cb0bf7c3470 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.975022729 +0000 UTC m=+37.067858659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-klrgt" (UID: "f46a5e6f-f333-4f27-a660-2cb0bf7c3470") : secret "samples-operator-tls" not found
Oct 30 18:37:10.986591 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:10.986537    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.075652    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.075722    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.075848    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.075970    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076025    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:15.076008388 +0000 UTC m=+37.168844311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076085    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076118    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:15.076107265 +0000 UTC m=+37.168943177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076179    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076190    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8457959978-p9454: secret "image-registry-tls" not found
Oct 30 18:37:11.076266 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.076218    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls podName:82214171-38b1-45d5-8c9b-794a142335fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:15.0762079 +0000 UTC m=+37.169043824 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls") pod "image-registry-8457959978-p9454" (UID: "82214171-38b1-45d5-8c9b-794a142335fc") : secret "image-registry-tls" not found
Oct 30 18:37:11.176800 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.176764    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:11.177105 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.177082    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:11.177269 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.177228    2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:37:11.177435 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:11.177335    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.177302545 +0000 UTC m=+65.270138457 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : secret "metrics-daemon-secret" not found
Oct 30 18:37:11.179917 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.179893    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:11.279360 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.279269    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:11.290557 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.290517    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:11.408044 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.407855    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:11.421291 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.421259    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:11.771036 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.769453    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kp5w5"]
Oct 30 18:37:11.772473 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:11.772431    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e6538b1_d3ff_4e67_81e0_e147ce5d03db.slice/crio-c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71 WatchSource:0}: Error finding container c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71: Status 404 returned error can't find the container with id c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71
Oct 30 18:37:11.783001 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:11.782976    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kps9t"]
Oct 30 18:37:11.785504 ip-10-0-132-168 kubenswrapper[2581]: W1030 18:37:11.785470    2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f258b8_0324_4e3a_85a1_50c4194f593d.slice/crio-cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7 WatchSource:0}: Error finding container cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7: Status 404 returned error can't find the container with id cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7
Oct 30 18:37:12.659737 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.659700    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kp5w5" event={"ID":"0e6538b1-d3ff-4e67-81e0-e147ce5d03db","Type":"ContainerStarted","Data":"c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71"}
Oct 30 18:37:12.661736 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.661667    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8" event={"ID":"e27601fc-9881-482b-9115-a169779efa5b","Type":"ContainerStarted","Data":"3aedc4fc90d43ce22b56809f65eae27e434f2343c5520626794a5353f8ea9f14"}
Oct 30 18:37:12.663269 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.663198    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kps9t" event={"ID":"16f258b8-0324-4e3a-85a1-50c4194f593d","Type":"ContainerStarted","Data":"cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7"}
Oct 30 18:37:12.668123 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.668103    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/0.log"
Oct 30 18:37:12.668205 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.668156    2581 generic.go:356] "Generic (PLEG): container finished" podID="808b5fa2-8623-4639-870a-be16d38cafeb" containerID="bdda26999c71adeb11032ceb774253fd7a5435dfae4490fa3fc83f933a26c140" exitCode=255
Oct 30 18:37:12.668205 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.668190    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" event={"ID":"808b5fa2-8623-4639-870a-be16d38cafeb","Type":"ContainerDied","Data":"bdda26999c71adeb11032ceb774253fd7a5435dfae4490fa3fc83f933a26c140"}
Oct 30 18:37:12.668503 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.668474    2581 scope.go:122] "RemoveContainer" containerID="bdda26999c71adeb11032ceb774253fd7a5435dfae4490fa3fc83f933a26c140"
Oct 30 18:37:12.707813 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:12.707767    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8" podStartSLOduration=179.308600923 podStartE2EDuration="3m3.707751491s" totalImagesPullingTime="4.399150568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:12.706857203 +0000 UTC m=+34.799693147" watchObservedRunningTime="2026-10-30 18:37:12.707751491 +0000 UTC m=+34.800587422"
Oct 30 18:37:13.022026 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.021988    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.160760 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.160649    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.350435 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.350330    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.371158 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.371134    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"]
Oct 30 18:37:13.405477 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.405452    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"]
Oct 30 18:37:13.405625 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.405563    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:13.484307 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.484230    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.498838 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.498585    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:13.498838 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.498716    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gwgkn\""
Oct 30 18:37:13.498838 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.498802    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:37:13.500789 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.500562    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2rd4\" (UniqueName: \"kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4\") pod \"migrator-66df5d496b-wfrjd\" (UID: \"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:13.601599 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.601559    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2rd4\" (UniqueName: \"kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4\") pod \"migrator-66df5d496b-wfrjd\" (UID: \"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:13.631657 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.624694    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2rd4\" (UniqueName: \"kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4\") pod \"migrator-66df5d496b-wfrjd\" (UID: \"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:13.631657 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.629502    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.675889 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.675183    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-mvhgb" event={"ID":"bf0111a4-adeb-451d-a0c2-17469dce5b90","Type":"ContainerStarted","Data":"e68823097d682ec7b08f72b174c2ceaf15248c1b02204d57dca4557de2b531bf"}
Oct 30 18:37:13.678019 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.677988    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:13.678513 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.678494    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/0.log"
Oct 30 18:37:13.678615 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.678530    2581 generic.go:356] "Generic (PLEG): container finished" podID="808b5fa2-8623-4639-870a-be16d38cafeb" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b" exitCode=255
Oct 30 18:37:13.678615 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.678597    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" event={"ID":"808b5fa2-8623-4639-870a-be16d38cafeb","Type":"ContainerDied","Data":"7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"}
Oct 30 18:37:13.678684 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.678653    2581 scope.go:122] "RemoveContainer" containerID="bdda26999c71adeb11032ceb774253fd7a5435dfae4490fa3fc83f933a26c140"
Oct 30 18:37:13.679009 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.678994    2581 scope.go:122] "RemoveContainer" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"
Oct 30 18:37:13.679224 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:13.679192    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:13.681887 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.681464    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9" event={"ID":"77ce2b35-2046-4b1a-bb36-7f49fe33f506","Type":"ContainerStarted","Data":"2d1b39522cd12e17761ea29bbb9f3e92c9db505c8bf467f6a4d804fbd48e0faa"}
Oct 30 18:37:13.683874 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.683805    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tkz7q" event={"ID":"cff1eb5a-0fc0-47ad-adff-12489a13e317","Type":"ContainerStarted","Data":"8a494505a796b22b9dcba5c0d4e4319d3c927f0537092454cdd01f698db3fbf1"}
Oct 30 18:37:13.724276 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.723613    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-mvhgb" podStartSLOduration=174.918649388 podStartE2EDuration="2m59.723595401s" totalImagesPullingTime="4.804946013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:13.722850051 +0000 UTC m=+35.815685983" watchObservedRunningTime="2026-10-30 18:37:13.723595401 +0000 UTC m=+35.816431333"
Oct 30 18:37:13.750749 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.750033    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:13.818676 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.818041    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tkz7q" podStartSLOduration=1.5198591700000001 podStartE2EDuration="6.818022468s" totalImagesPullingTime="5.298163298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:13.772462661 +0000 UTC m=+35.865298591" watchObservedRunningTime="2026-10-30 18:37:13.818022468 +0000 UTC m=+35.910858400"
Oct 30 18:37:13.818676 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.818395    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9" podStartSLOduration=179.852139388 podStartE2EDuration="3m4.818386934s" totalImagesPullingTime="4.966247546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:13.816461935 +0000 UTC m=+35.909297867" watchObservedRunningTime="2026-10-30 18:37:13.818386934 +0000 UTC m=+35.911222866"
Oct 30 18:37:13.856483 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.854608    2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-p9vkt"]
Oct 30 18:37:13.876801 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.875533    2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:13.881137 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.880956    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9vkt"]
Oct 30 18:37:13.940818 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.940745    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:13.956733 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.956688    2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:37:13.983045 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.983024    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5w9ll\""
Oct 30 18:37:13.983184 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:13.983084    2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:37:14.006359 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.006324    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.006490 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.006434    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.006537 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.006509    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.006582 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.006567    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.006653 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.006638    2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm9qd\" (UniqueName: \"kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.107684 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.107647    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.107734    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9qd\" (UniqueName: \"kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.107765    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.107857    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.107915    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.108007    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:37:14.108077 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.108067    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.60804987 +0000 UTC m=+36.700885786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : configmap "kube-rbac-proxy" not found
Oct 30 18:37:14.108549 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.108373    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:14.108549 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.108377    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.108549 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.108429    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:14.608411314 +0000 UTC m=+36.701247223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:14.108662 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.108636    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.124454 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.124424    2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"]
Oct 30 18:37:14.132121 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.132069    2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9qd\" (UniqueName: \"kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.147202 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.147173    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:14.291488 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.291409    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:14.531443 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.531406    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.612698    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.612812    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.612928    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.612984    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:15.612966444 +0000 UTC m=+37.705802355 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : configmap "kube-rbac-proxy" not found
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.613372    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:14.613506 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.613425    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:15.613409207 +0000 UTC m=+37.706245133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:14.673818 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.673721    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:14.689155 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.689122    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd" event={"ID":"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493","Type":"ContainerStarted","Data":"77ee943e51c455b302ebb75e552facc0bdbbd74fe771a713383f94c50d9bdb22"}
Oct 30 18:37:14.691227 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.691194    2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:14.691859 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.691652    2581 scope.go:122] "RemoveContainer" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"
Oct 30 18:37:14.691971 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:14.691870    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:14.693974 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.693523    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc" event={"ID":"de24546a-df74-492e-98f2-6dcf0d466108","Type":"ContainerStarted","Data":"42f0d64a94fe5bf8e8c858c21ba479810c5b6e0b8181062c2c70cf1cde9b5103"}
Oct 30 18:37:14.778596 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:14.778535    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc" podStartSLOduration=176.130376034 podStartE2EDuration="3m1.778516145s" totalImagesPullingTime="5.648140111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:14.777814955 +0000 UTC m=+36.870650887" watchObservedRunningTime="2026-10-30 18:37:14.778516145 +0000 UTC m=+36.871352077"
Oct 30 18:37:15.017239 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.017205    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:15.017378 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.017276    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:15.017378 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.017327    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:15.017378 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.017363    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:15.017542 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017400    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.017377429 +0000 UTC m=+45.110213351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:15.017542 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017457    2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:15.017542 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.017499    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:15.017542 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017539    2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017594    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017604    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5bbf88fc59-82jt6: secret "image-registry-tls" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017633    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.017618347 +0000 UTC m=+45.110454271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : secret "router-metrics-certs-default" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017654    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls podName:50f0c5e2-b263-4047-b980-20b8ebb07a79 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.017645091 +0000 UTC m=+45.110481006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls") pod "image-registry-5bbf88fc59-82jt6" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79") : secret "image-registry-tls" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017656    2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017687    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.017672097 +0000 UTC m=+45.110508020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:15.017742 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.017719    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls podName:f46a5e6f-f333-4f27-a660-2cb0bf7c3470 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.017709386 +0000 UTC m=+45.110545314 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-klrgt" (UID: "f46a5e6f-f333-4f27-a660-2cb0bf7c3470") : secret "samples-operator-tls" not found
Oct 30 18:37:15.119050 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.119017    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:15.119545 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.119109    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:15.119545 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.119294    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:15.119545 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119490    2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:15.119724 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119576    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.119558106 +0000 UTC m=+45.212394018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:15.119724 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119650    2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:15.119724 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119689    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.119676639 +0000 UTC m=+45.212512565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:15.119866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119752    2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 30 18:37:15.119866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119764    2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8457959978-p9454: secret "image-registry-tls" not found
Oct 30 18:37:15.119866 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.119796    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls podName:82214171-38b1-45d5-8c9b-794a142335fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.119786015 +0000 UTC m=+45.212621937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls") pod "image-registry-8457959978-p9454" (UID: "82214171-38b1-45d5-8c9b-794a142335fc") : secret "image-registry-tls" not found
Oct 30 18:37:15.147613 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.147583    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:15.290593 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.290520    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:15.623500 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.623468    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:15.623656 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.623596    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:15.623656 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.623607    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:37:15.623773 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.623686    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:17.623663019 +0000 UTC m=+39.716498942 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : configmap "kube-rbac-proxy" not found
Oct 30 18:37:15.623773 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.623708    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:15.623773 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:15.623768    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:17.62375147 +0000 UTC m=+39.716587396 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:15.700134 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.700101    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd" event={"ID":"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493","Type":"ContainerStarted","Data":"e5e18eaaa6a8099b77b5aebaa1f923361df7ca4db6205c920983dba95b6e3c75"}
Oct 30 18:37:15.701816 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.701780    2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kps9t" event={"ID":"16f258b8-0324-4e3a-85a1-50c4194f593d","Type":"ContainerStarted","Data":"6551b96105fb3637e3a1df161fe00d8381848909d491b77a1c294b41051eda44"}
Oct 30 18:37:15.742219 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.742167    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd" podStartSLOduration=1.306750823 podStartE2EDuration="2.742148145s" totalImagesPullingTime="1.435397322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:15.741078341 +0000 UTC m=+37.833914273" watchObservedRunningTime="2026-10-30 18:37:15.742148145 +0000 UTC m=+37.834984077"
Oct 30 18:37:15.785191 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.785146    2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kps9t" podStartSLOduration=34.634607647 podStartE2EDuration="37.78513294s" totalImagesPullingTime="3.150525293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:15.78415364 +0000 UTC m=+37.876989571" watchObservedRunningTime="2026-10-30 18:37:15.78513294 +0000 UTC m=+37.877968871"
Oct 30 18:37:15.922090 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:15.922008    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:16.408570 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:16.408451    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:16.547192 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:16.547160    2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:16.705646 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:16.705577    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:17.588482 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:17.588447    2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:17.588866 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:17.588505    2581 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:17.588866 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:17.588816    2581 scope.go:122] "RemoveContainer" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"
Oct 30 18:37:17.588993 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:17.588975    2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:17.645893 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:17.645867    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:17.646014 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:17.645996    2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:17.646083 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:17.646001    2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 18:37:17.647072 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:17.646614    2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:17.647072 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:17.646732    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.646701326 +0000 UTC m=+43.739537256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:17.647072 ip-10-0-132-168 kubenswrapper[2581]: E1030 18:37:17.646828    2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.646797934 +0000 UTC m=+43.739633858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : configmap "kube-rbac-proxy" not found
Oct 30 18:37:18.161695 ip-10-0-132-168 kubenswrapper[2581]: I1030 18:37:18.161527    2581 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:37:18.161760 ip-10-0-132-168 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:37:18.181590 ip-10-0-132-168 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:37:18.181759 ip-10-0-132-168 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:37:18.181806 ip-10-0-132-168 systemd[1]: kubelet.service: Consumed 3.002s CPU time.
Oct 30 18:37:18.193743 ip-10-0-132-168 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:37:18.309317 ip-10-0-132-168 kubenswrapper[4938]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:37:18.309317 ip-10-0-132-168 kubenswrapper[4938]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:37:18.309317 ip-10-0-132-168 kubenswrapper[4938]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:37:18.309317 ip-10-0-132-168 kubenswrapper[4938]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:37:18.311435 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311359    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.311435 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311434    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.311435 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311439    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311443    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311446    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311449    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311453    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311456    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311460    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311463    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311466    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311469    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311472    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311475    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311477    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311486    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311489    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311492    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311494    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311497    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311500    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.311541 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311502    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311505    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311508    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311511    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311515    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311517    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311520    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311522    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311525    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311528    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311531    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311533    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311536    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311539    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311541    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311544    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311547    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311550    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311552    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311555    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.312029 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311558    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311560    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311563    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311565    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311568    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311570    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311573    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311575    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311578    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311581    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311583    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311586    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311589    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311591    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311594    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311597    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311600    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311603    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311605    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311608    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.312552 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311610    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311613    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311615    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311618    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311621    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311624    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311626    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311629    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311632    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311635    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311639    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311643    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311645    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311648    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311651    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311654    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311656    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311659    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311662    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.313092 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311665    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311670    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311673    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311677    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311681    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311684    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311687    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311690    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311693    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311696    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311698    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311701    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311703    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311706    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311709    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311711    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311714    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311716    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311723    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.313586 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311727    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311730    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311733    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311736    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311738    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311741    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.311743    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312502    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312509    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312512    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312515    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312517    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312520    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312523    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312526    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312528    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312531    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312537    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312539    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.314066 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312543    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312546    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312549    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312551    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312554    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312557    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312560    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312563    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312565    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312568    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312571    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312573    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312576    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312579    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312582    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312584    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312587    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312589    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312592    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.314556 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312595    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312597    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312600    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312603    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312605    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312608    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312611    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312614    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312616    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312619    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312621    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312624    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312627    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312630    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312632    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312636    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312638    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312641    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312643    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312646    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.315046 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312648    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312651    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312653    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312656    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312661    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312664    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312667    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312670    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312673    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312676    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312678    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312681    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312683    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312686    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312688    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312691    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312693    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312696    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312698    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.315572 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312701    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312704    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312707    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312710    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312713    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312715    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312718    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312721    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312724    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312726    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312729    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312731    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312734    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312737    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312740    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312743    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312745    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312748    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312750    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312753    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.316045 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312755    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312759    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312762    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312764    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312767    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312770    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312772    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312775    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312777    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312780    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312782    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312787    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312791    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312793    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312796    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.312799    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313261    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313267    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313270    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.316543 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313273    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313277    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313280    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313283    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313286    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313288    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313291    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313294    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313296    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313299    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313301    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313304    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313307    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313310    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313313    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313315    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313318    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313320    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313323    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313326    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.317020 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313328    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313331    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313334    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313336    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313339    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313341    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313344    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313347    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313349    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313352    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313354    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313357    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313360    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313362    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313365    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313368    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313371    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313374    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313376    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.318014 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313381    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313384    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313388    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313390    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313393    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313398    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313400    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313403    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313406    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313409    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313411    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313414    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313417    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313419    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313422    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313424    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313427    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313429    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313432    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313435    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.318691 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313437    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313440    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313443    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313445    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313448    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313451    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313455    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313458    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313460    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313463    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313465    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313468    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313470    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313473    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313475    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313478    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313480    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313484    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313487    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.319201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313489    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313492    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313495    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313498    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313500    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313503    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313505    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313508    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313510    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313513    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313515    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313518    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313520    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313523    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313525    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313528    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313530    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313533    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313535    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313540    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313542    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.319724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313545    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313547    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313550    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.313552    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313642    4938 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313649    4938 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313655    4938 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313660    4938 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313664    4938 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313669    4938 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313673    4938 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313678    4938 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313681    4938 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313684    4938 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313688    4938 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313691    4938 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313695    4938 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313698    4938 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313701    4938 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313704    4938 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313707    4938 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313710    4938 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313714    4938 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:37:18.320311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313717    4938 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313720    4938 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313723    4938 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313727    4938 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313730    4938 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313734    4938 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313737    4938 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313741    4938 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313745    4938 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313748    4938 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313751    4938 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313754    4938 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313757    4938 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313761    4938 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313764    4938 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313767    4938 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313770    4938 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313773    4938 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313777    4938 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313782    4938 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313785    4938 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313788    4938 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313791    4938 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313794    4938 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:37:18.320923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313798    4938 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313802    4938 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313805    4938 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313809    4938 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313812    4938 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313815    4938 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313818    4938 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313820    4938 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313824    4938 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313827    4938 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313829    4938 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313833    4938 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313837    4938 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313840    4938 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313843    4938 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313848    4938 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313851    4938 flags.go:64] FLAG: --help="false"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313854    4938 flags.go:64] FLAG: --hostname-override="ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313857    4938 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313860    4938 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313863    4938 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313867    4938 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313870    4938 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313873    4938 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:37:18.321533 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313876    4938 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313879    4938 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313883    4938 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313886    4938 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313890    4938 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313893    4938 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313895    4938 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313898    4938 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313901    4938 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313904    4938 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313908    4938 flags.go:64] FLAG: --lock-file=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313911    4938 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313914    4938 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313917    4938 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313922    4938 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313926    4938 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313929    4938 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313932    4938 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313935    4938 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313938    4938 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313942    4938 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313945    4938 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313949    4938 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313953    4938 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313957    4938 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:37:18.322133 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313960    4938 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313964    4938 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313967    4938 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313970    4938 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313973    4938 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313976    4938 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313979    4938 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313985    4938 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313988    4938 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313993    4938 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313996    4938 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.313999    4938 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314002    4938 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314005    4938 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314008    4938 flags.go:64] FLAG: --port="10250"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314011    4938 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314014    4938 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0326a226477f2e310"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314018    4938 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314021    4938 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314024    4938 flags.go:64] FLAG: --register-node="true"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314027    4938 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314031    4938 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314034    4938 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314037    4938 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314040    4938 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314043    4938 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:37:18.322761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314046    4938 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314049    4938 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314052    4938 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314055    4938 flags.go:64] FLAG: --runonce="false"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314060    4938 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314063    4938 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314066    4938 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314070    4938 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314073    4938 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314076    4938 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314079    4938 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314082    4938 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314085    4938 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314088    4938 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314091    4938 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314096    4938 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314099    4938 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314102    4938 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314105    4938 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314110    4938 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314113    4938 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314116    4938 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314120    4938 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314123    4938 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314128    4938 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:37:18.323473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314132    4938 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314135    4938 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314138    4938 flags.go:64] FLAG: --v="2"
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314142    4938 flags.go:64] FLAG: --version="false"
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314146    4938 flags.go:64] FLAG: --vmodule=""
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314150    4938 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314153    4938 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314287    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314291    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314293    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314296    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314301    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314304    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314306    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314309    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314312    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314314    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314317    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314320    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314322    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314325    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.324085 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314328    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314333    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314337    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314340    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314343    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314345    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314348    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314351    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314353    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314356    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314359    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314362    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314364    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314367    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314369    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314372    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314375    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314378    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314380    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.324783 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314383    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314385    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314388    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314391    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314394    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314397    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314399    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314402    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314405    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314407    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314410    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314413    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314415    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314418    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314422    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314424    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314427    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314429    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314432    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.325350 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314435    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314438    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314442    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314445    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314448    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314451    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314454    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314457    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314460    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314462    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314465    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314467    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314470    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314472    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314475    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314477    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314482    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314484    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314487    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.326179 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314489    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314492    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314495    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314497    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314500    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314502    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314505    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314508    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314512    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314514    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314517    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314519    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314522    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314525    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314527    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314530    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314532    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314535    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314538    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314541    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.327073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314544    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314546    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314549    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314552    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314554    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314557    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314560    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314563    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314565    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314569    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314571    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314574    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314577    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314579    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314582    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.314614    4938 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:37:18.327780 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314648    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314653    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314655    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314658    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314661    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314665    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314668    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314671    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314673    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314676    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314679    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314682    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314684    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314687    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314690    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314693    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314695    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314698    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314701    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314704    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.328241 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314707    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314709    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314712    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314714    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314717    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314719    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314722    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314724    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314727    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314730    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314732    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314734    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314737    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314740    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314742    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314745    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314748    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314752    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314755    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314757    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.328828 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314760    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314762    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314767    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314771    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314774    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314777    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314780    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314783    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314785    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314788    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314791    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314793    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314796    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314798    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314801    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314804    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314806    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314809    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314811    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.329344 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314814    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314817    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314819    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314822    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314824    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314827    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314830    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314833    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314836    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314839    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314842    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314845    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314848    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314851    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314853    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314856    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314858    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314861    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314864    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.329811 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314866    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314869    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314873    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314876    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314879    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314883    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314886    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314888    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314892    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314894    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314897    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314899    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314902    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314905    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314907    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314910    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314912    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314915    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314917    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.330310 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314920    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314922    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314925    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314928    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314932    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314934    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314937    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314939    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.314943    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.319820    4938 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.319835    4938 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.319853    4938 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.319858    4938 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319913    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319918    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319921    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319925    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319928    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319930    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.330788 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319933    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319936    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319939    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319941    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319944    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319947    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319949    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319952    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319955    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319958    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319960    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319963    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319966    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319969    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319971    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319974    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319976    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319979    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319982    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.331412 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319984    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319987    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319990    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319993    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319997    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.319999    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320002    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320005    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320007    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320010    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320013    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320015    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320018    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320021    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320024    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320026    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320029    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320032    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320034    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320037    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.332213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320039    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320042    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320045    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320047    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320050    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320052    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320055    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320058    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320060    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320063    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320066    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320069    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320071    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320074    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320077    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320079    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320082    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320085    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320088    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320091    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.332976 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320093    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320096    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320099    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320103    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320107    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320111    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320114    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320116    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320119    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320122    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320125    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320129    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320133    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320136    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320139    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320142    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320145    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320148    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.333724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320151    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320154    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320156    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320159    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320162    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320164    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320167    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320170    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320173    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320175    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320178    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320181    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320184    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320186    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320189    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320192    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320194    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320197    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320199    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320202    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.334213 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320205    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320208    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320211    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.320239    4938 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320388    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320393    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320396    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320399    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320402    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320405    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320408    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320411    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320414    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320417    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320419    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.334726 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320422    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320425    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320427    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320430    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320433    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320436    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320438    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320441    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320444    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320446    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320449    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320452    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320454    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320457    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320459    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320462    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320464    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320467    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320472    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.335096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320476    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320479    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320482    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320484    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320487    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320489    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320492    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320495    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320499    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320501    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320504    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320506    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320509    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320512    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320514    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320517    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320520    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320522    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320525    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320527    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.335576 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320533    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320535    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320538    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320540    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320544    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320548    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320552    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320555    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320558    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320561    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320563    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320566    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320569    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320571    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320573    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320576    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320579    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320581    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320584    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320587    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.336073 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320589    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320596    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320599    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320601    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320604    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320606    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320609    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320611    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320614    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320617    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320619    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320622    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320626    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320628    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320631    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320633    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320636    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320638    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320641    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320644    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.336756 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320647    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320649    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320652    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320654    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320657    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320660    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320662    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320665    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320667    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320670    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320673    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320675    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320677    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320680    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320683    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320686    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.337272 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.320709    4938 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320741    4938 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320745    4938 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320748    4938 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320751    4938 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320753    4938 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320760    4938 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320763    4938 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320767    4938 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320770    4938 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320772    4938 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320775    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320777    4938 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320780    4938 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320783    4938 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320785    4938 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320788    4938 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320790    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320793    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320796    4938 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:37:18.337672 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320799    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320801    4938 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320804    4938 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320806    4938 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320809    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320812    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320814    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320817    4938 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320819    4938 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320822    4938 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320825    4938 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320828    4938 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320830    4938 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320833    4938 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320836    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320838    4938 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320841    4938 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320843    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320846    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:37:18.338197 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320848    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320853    4938 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320855    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320858    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320860    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320863    4938 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320865    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320868    4938 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320870    4938 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320873    4938 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320876    4938 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320879    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320881    4938 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320884    4938 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320886    4938 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320889    4938 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320892    4938 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320894    4938 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320897    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:37:18.338693 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320899    4938 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320902    4938 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320905    4938 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320907    4938 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320909    4938 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320912    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320915    4938 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320919    4938 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320922    4938 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320926    4938 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320930    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320933    4938 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320936    4938 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320939    4938 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320943    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320946    4938 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320948    4938 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320951    4938 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320953    4938 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:37:18.339178 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320955    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320958    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320961    4938 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320963    4938 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320966    4938 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320969    4938 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320971    4938 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320974    4938 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320977    4938 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320979    4938 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320982    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320984    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320987    4938 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320991    4938 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320994    4938 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320997    4938 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.320999    4938 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321002    4938 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321005    4938 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321007    4938 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:37:18.339771 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321010    4938 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321013    4938 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321016    4938 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321018    4938 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321021    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321025    4938 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321027    4938 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321032    4938 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321034    4938 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:18.321037    4938 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.321084    4938 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322088    4938 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322148    4938 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322351    4938 server.go:1014] "Starting client certificate rotation"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322487    4938 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322543    4938 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:38 +0000 UTC" deadline="2028-08-04 13:14:53.679664884 +0000 UTC"
Oct 30 18:37:18.340761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322569    4938 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15450h37m35.35709784s"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.322836    4938 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.323923    4938 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.324975    4938 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.327167    4938 log.go:25] "Validated CRI v1 image API"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.327918    4938 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:37:18.341500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.329708    4938 fs.go:135] Filesystem UUIDs: map[09cf16da-8ed1-49a7-bc4e-05b1176fba13:/dev/nvme0n1p3 64eb4ad3-b8fc-4c78-aef2-c65c38169ad6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 18:37:18.341804 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.329729    4938 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d/userdata/shm major:0 minor:221 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28ed2be66f04084c9eca971198c00b98fd3e89a525b0a5a2643f3660dfa6f62e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28ed2be66f04084c9eca971198c00b98fd3e89a525b0a5a2643f3660dfa6f62e/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3af857443715187d65401c9b3b5941a739ac0b2307510d8b40a1212945eda39e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3af857443715187d65401c9b3b5941a739ac0b2307510d8b40a1212945eda39e/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/77ee943e51c455b302ebb75e552facc0bdbbd74fe771a713383f94c50d9bdb22/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/77ee943e51c455b302ebb75e552facc0bdbbd74fe771a713383f94c50d9bdb22/userdata/shm major:0 minor:334 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280/userdata/shm major:0 minor:241 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a31ee4704c1a7f3a9b874a489ba8eb27cea9c90cc4571ce6caf8ea0a06f1f6d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a31ee4704c1a7f3a9b874a489ba8eb27cea9c90cc4571ce6caf8ea0a06f1f6d3/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a403a38582bc34a4a061361dd926dd85c83ef5e684d6394a57bba53043fd5140/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a403a38582bc34a4a061361dd926dd85c83ef5e684d6394a57bba53043fd5140/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ad1dd7893496396ea9e505450a2a22e9c4869b88153f21bef4f1b0935b4c6b2b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ad1dd7893496396ea9e505450a2a22e9c4869b88153f21bef4f1b0935b4c6b2b/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f89ad751c0a5eec6f927bbae97c6888f6fcba86508d4526971315bdb0ec62bc5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f89ad751c0a5eec6f927bbae97c6888f6fcba86508d4526971315bdb0ec62bc5/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e6538b1-d3ff-4e67-81e0-e147ce5d03db/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/0e6538b1-d3ff-4e67-81e0-e147ce5d03db/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13252367-d85f-4a6a-9eae-2e22cfbb5c4f/volumes/kubernetes.io~projected/kube-api-access-gqnln:{mountpoint:/var/lib/kubelet/pods/13252367-d85f-4a6a-9eae-2e22cfbb5c4f/volumes/kubernetes.io~projected/kube-api-access-gqnln major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16f258b8-0324-4e3a-85a1-50c4194f593d/volumes/kubernetes.io~projected/kube-api-access-swtrr:{mountpoint:/var/lib/kubelet/pods/16f258b8-0324-4e3a-85a1-50c4194f593d/volumes/kubernetes.io~projected/kube-api-access-swtrr major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f/volumes/kubernetes.io~projected/kube-api-access-2ng5d:{mountpoint:/var/lib/kubelet/pods/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f/volumes/kubernetes.io~projected/kube-api-access-2ng5d major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493/volumes/kubernetes.io~projected/kube-api-access-c2rd4:{mountpoint:/var/lib/kubelet/pods/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493/volumes/kubernetes.io~projected/kube-api-access-c2rd4 major:0 minor:310 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/279b261c-91e6-427e-9e19-3f1a42e60e76/volumes/kubernetes.io~projected/kube-api-access-snl4j:{mountpoint:/var/lib/kubelet/pods/279b261c-91e6-427e-9e19-3f1a42e60e76/volumes/kubernetes.io~projected/kube-api-access-snl4j major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/310d68ba-e235-4caf-ba78-c1940e74354c/volumes/kubernetes.io~projected/kube-api-access-nx997:{mountpoint:/var/lib/kubelet/pods/310d68ba-e235-4caf-ba78-c1940e74354c/volumes/kubernetes.io~projected/kube-api-access-nx997 major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d127029-258f-4c97-a9b8-d2cf85123890/volumes/kubernetes.io~projected/kube-api-access-2n7l9:{mountpoint:/var/lib/kubelet/pods/4d127029-258f-4c97-a9b8-d2cf85123890/volumes/kubernetes.io~projected/kube-api-access-2n7l9 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/kube-api-access-ft5kv:{mountpoint:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/kube-api-access-ft5kv major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc/volumes/kubernetes.io~projected/kube-api-access-cxfsz:{mountpoint:/var/lib/kubelet/pods/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc/volumes/kubernetes.io~projected/kube-api-access-cxfsz major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~projected/kube-api-access-vfppz:{mountpoint:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~projected/kube-api-access-vfppz major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/default-certificate major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/stats-auth major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7265c58c-0d6e-4fe4-98af-9db6238dd3b8/volumes/kubernetes.io~projected/kube-api-access-ff5qc:{mountpoint:/var/lib/kubelet/pods/7265c58c-0d6e-4fe4-98af-9db6238dd3b8/volumes/kubernetes.io~projected/kube-api-access-ff5qc major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~projected/kube-api-access-bxbmh:{mountpoint:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~projected/kube-api-access-bxbmh major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/748b974a-4a23-4a7a-949b-3dc6f972ea61/volumes/kubernetes.io~projected/kube-api-access-rt6qt:{mountpoint:/var/lib/kubelet/pods/748b974a-4a23-4a7a-949b-3dc6f972ea61/volumes/kubernetes.io~projected/kube-api-access-rt6qt major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~projected/kube-api-access-2fhfx:{mountpoint:/var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~projected/kube-api-access-2fhfx major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ba4b2b7-3196-4769-aaa9-8c40dd282893/volumes/kubernetes.io~projected/kube-api-access-vjw94:{mountpoint:/var/lib/kubelet/pods/7ba4b2b7-3196-4769-aaa9-8c40dd282893/volumes/kubernetes.io~projected/kube-api-access-vjw94 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~projected/kube-api-access-k87cx:{mountpoint:/var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~projected/kube-api-access-k87cx major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/kube-api-access-5zjrb:{mountpoint:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/kube-api-access-5zjrb major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~projected/kube-api-access-zwzn2:{mountpoint:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~projected/kube-api-access-zwzn2 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8feb3890-bb59-4850-a7cf-7c53c9048ad1/volumes/kubernetes.io~projected/kube-api-access-xm9qd:{mountpoint:/var/lib/kubelet/pods/8feb3890-bb59-4850-a7cf-7c53c9048ad1/volumes/kubernetes.io~projected/kube-api-access-xm9qd major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/90889eb1-58bd-4e0a-802c-3156bbe5ef2f/volumes/kubernetes.io~projected/kube-api-access-rw5zx:{mountpoint:/var/lib/kubelet/pods/90889eb1-58bd-4e0a-802c-3156bbe5ef2f/volumes/kubernetes.io~projected/kube-api-access-rw5zx major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~projected/kube-api-access-m7jhr:{mountpoint:/var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~projected/kube-api-access-m7jhr major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~secret/serving-cert major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cca6e1a6-ea37-482c-9c61-1a7ac0259030/volumes/kubernetes.io~projected/kube-api-access-56cnn:{mountpoint:/var/lib/kubelet/pods/cca6e1a6-ea37-482c-9c61-1a7ac0259030/volumes/kubernetes.io~projected/kube-api-access-56cnn major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf3e1e65-7ce3-4b20-9923-98c273dd915f/volumes/kubernetes.io~projected/kube-api-access-nvhz7:{mountpoint:/var/lib/kubelet/pods/cf3e1e65-7ce3-4b20-9923-98c273dd915f/volumes/kubernetes.io~projected/kube-api-access-nvhz7 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~projected/kube-api-access-5wwgx:{mountpoint:/var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~projected/kube-api-access-5wwgx major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~secret/cert major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de24546a-df74-492e-98f2-6dcf0d466108/volumes/kubernetes.io~projected/kube-api-access-8l96t:{mountpoint:/var/lib/kubelet/pods/de24546a-df74-492e-98f2-6dcf0d466108/volumes/kubernetes.io~projected/kube-api-access-8l96t major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~projected/kube-api-access-mmgkh:{mountpoint:/var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~projected/kube-api-access-mmgkh major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e69d8d91-a4a3-412d-ab1b-d9a8409580fa/volumes/kubernetes.io~projected/kube-api-access-5wrq2:{mountpoint:/var/lib/kubelet/pods/e69d8d91-a4a3-412d-ab1b-d9a8409580fa/volumes/kubernetes.io~projected/kube-api-access-5wrq2 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f46a5e6f-f333-4f27-a660-2cb0bf7c3470/volumes/kubernetes.io~projected/kube-api-access-jnl2m:{mountpoint:/var/lib/kubelet/pods/f46a5e6f-f333-4f27-a660-2cb0bf7c3470/volumes/kubernetes.io~projected/kube-api-access-jnl2m major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fca72a21-1b7a-4005-9baa-970f86ebd652/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/fca72a21-1b7a-4005-9baa-970f86ebd652/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/72fc0d005bdcd1d1a8e141a4ec4b4b88c488d1ab2a833fa3441efd1ee12c8e47/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/eef0ad08f91aa8d1f76fc97b83ea5047c1c175cec44b403341e27b2962cfb4aa/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/b629f7872918cb20608787afdd2f9a98380299f94a2b84db747db4d8d56936fb/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/efd322becfc0423d8595d51a61dcb927d5e4056e2ff2300ffa548a0e0082f8ec/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/7d617671158edbf517b5f13a287c335662c47e7e3bdcb8598f0bc38eaec0614e/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/08931df1b1b3756ac9fc1a15513b1fb6bff2acc503a1d88f53d8991994544786/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/7733d3516cabfad330e7ce0342312ff68e7d92565a68fe4bec485760c08bd376/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/8ea03683a34247834469f6ddaa36f850332f5d64e67d591ccb0f65613239f6b3/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/a8e7ef88513fceb451ab3c414c0542a109eb54a89f81766ed2564e8e178dd76a/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/f631de454c2d5c1b4760f04e0233422c28011bfd0e7175ae0ec355f4fe5f1881/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/fbd67907ac098db6698ce470942f200aff8665fcf8c566848cabe11f22c67ff6/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/93c2eb7e6fc821d47a2ebc5a1edee71d279ccbef9370fa99074902823679b0df/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/86ffb44a4df2757aab87f4b57c1d643f9eaa4486ce759efc5a1a9ef80d92daab/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/d7552e593d4e87ec6fbb267451d9b937ac5ff94c0493f3c4129adff0ebb19ebc/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/79620be08c4d79276174f3d2796670f96adc23edc650e6945b62e65f1d4481bf/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/d260ce152b452bffc9e91dbb042a7bc4b89407b5c87259f82ead5859bd407a85/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/148d23704a532bf6b664d24333962a8cce910f140dd5c5f5f1f73fbd7cbeaab2/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/d14cb8ac5e5f05877a5976b5a7a56d5328f77b5266348811a7f49ee10de007fb/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/d8a140fe5d8b6ff9695305a62fd44cd4018f536854a54eaf920f7684f85eccbe/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/811352f1fcd00800068a55b12585e16aa86f3e51d11b4cb7bd2d72d3f6ed210e/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/383072763aade76154437a593d7975ee66af6406eb5058f739f966b99e05eda9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/94f1276aff82b0c2010db25e08f36362d8cd5d8dd228c7591727dcdc79c40a8d/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/5f01d9c47824e09f1bea9e89d2541b872b66d572c836127c54f0fc0e95f4b155/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c84bfc45c982a9ea3608301e91747289706228afa0cecb55794ffa11d1aa68f7/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/35bb35738b4667fd0cbb78caa69e508e9d9fbb02d989c2e342e541e2463f1acb/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/c6865737891e1514aec3116d91622297d100b8e15bb4af84a9510dc769e94c34/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/99c417d3095d8b66150c7ea3d2e2c7b8bf62e7b21abbfda0161c2ca390b98e58/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/7ca13da704a1652e38b965333ffe69ae4f8892ecf0bfcbb4b0441baa0e01e356/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/eccd35800f8483f4ddfd00d414686891a789878f7171bd3d90b0f1c919920565/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/f04331f91815ff870139404a9c43b93e40a499e617694ca94290262e77e0464e/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/b8e9000163949e20f3ec77494cdf993fa1d1bda5d7c000ca52ccf4a12f519d61/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/87aea7280f6f83ab8f32f0d6ee2a82961872f7f012dafe2ca478b9eb5d1983e7/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/9f0e3bb651779e914cb4f4b928c2c1211cd480954f83346b7fc3cef9d5b812b2/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/b71244cbdd961d010dd6ca90987a70ee940fa04872ae3b935ce213b796b5bb5a/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/55034ffaf98bf1f8e685806b0108c737cdd6dd435e6b33649e28387a0e722825/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/ccabe0c1d21851ad9e47c809653c7eaed1ff0734655086d2351a5a9fe5ba9a26/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/166f0d40c89774320fd7e6ed1650ab8907c8f601e58c49166390294204eb2ee1/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/f0402c73500830828f7255edcf43595c001ff30dfaffdbd3645ac922a0d04279/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/420294bd88476cdc1f5d12dc1cc92860776ab3521723fe9884dad5cffd16711a/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/b087236a47465452ea4fc18d09bc1e196908cf58ef2d781a3ac7ee69afbef616/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/795d202e81ceed4b8dc1d87e23b8672025ba91fb4697a64d8330441f84c5d6a6/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-319:{mountpoint:/var/lib/containers/storage/overlay/5d426e9b3ba667bb0d7824e3169e5b29d31d009ccfef003c0970547592f4f9f3/merged major:0 minor:319 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/f00fff3ac8ca824d2421995b7c803f70b63d1a4aed7fc026dbc6919181088945/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-338:{mountpoint:/var/lib/containers/storage/overlay/185154e6e6a03ff1f1d4ed3fcef0f276e123aa545b39ea167bff9374577b6926/merged major:0 minor:338 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/3305c4de395f605e6a9bf882dd19189a13fc1f68d411e46cfc24c57b84596118/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/b173713f3b882b301c23297bcde91d0413b99f4767531179e36455606121fd3c/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/9ebca1feeaeca6f3511a07ebb0569c1d3ab54f0e709fabaf93e250772982424b/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/bb2b7690eaeb2cd8874a800f2b592a4e5a55c96a9c7abaf1ac900a4f72c17d77/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ad6f34396ebb93880537adb81f55ba377ee3b017bd16fe834e630bd556854ce4/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/53e0d4813654dadefe7f21ac8ed1b26f92d0fbd2bc8a8a8aac2e4673b280b603/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/f8d998ab21702573d2eb22729f91f25e73e53d906b72298ae928f599fd3cdc5a/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/f74824874f3737e8f1502e32bfa51916c5b3d1cd08118dc41dca8b820e413d48/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/1629e0e9544ebf6eb796596490b54da6b390751598bf1ae79bb9d14f1977ab29/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/0d083370bfa29a6eff127aa5f87278b7813ef38dad8cf36f746bfbeb81a89180/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/d75366a29780e13c1b341fdd91ac3bd2bec83336406fec8dafbbfa7bcd62cfb8/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/61fe640580c6c77e85bc80ab8222a32c740896c1f600c781a586bf7498b5ce93/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.338377    4938 manager.go:217] Machine: {Timestamp:2026-10-30 18:37:18.337899948 +0000 UTC m=+0.097908403 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100230 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2545cafeddc65595ea17506f692915 SystemUUID:ec2545ca-fedd-c655-95ea-17506f692915 BootID:993a3fea-3e8c-4d1e-b98f-4086d2397eee Filesystems:[{Device:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7ba4b2b7-3196-4769-aaa9-8c40dd282893/volumes/kubernetes.io~projected/kube-api-access-vjw94 DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/16f258b8-0324-4e3a-85a1-50c4194f593d/volumes/kubernetes.io~projected/kube-api-access-swtrr DeviceMajor:0 DeviceMinor:293 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493/volumes/kubernetes.io~projected/kube-api-access-c2rd4 DeviceMajor:0 DeviceMinor:310 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-319 DeviceMajor:0 DeviceMinor:319 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:207 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-338 DeviceMajor:0 DeviceMinor:338 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cca6e1a6-ea37-482c-9c61-1a7ac0259030/volumes/kubernetes.io~projected/kube-api-access-56cnn DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~projected/kube-api-access-5wwgx DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~projected/kube-api-access-ft5kv DeviceMajor:0 DeviceMinor:213 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/90889eb1-58bd-4e0a-802c-3156bbe5ef2f/volumes/kubernetes.io~projected/kube-api-access-rw5zx DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/218f66fbe7bfb79d0df4dd49b7374ac0036f8551c60a1e0e110faa493775eb12/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/684df514525aa4ba2865bfaccad1414ea9bcb606a99905d6579b4436f9143073/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f/volumes/kubernetes.io~projected/kube-api-access-2ng5d DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~projected/kube-api-access-vfppz DeviceMajor:0 DeviceMinor:212 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4230cdd444de3e8988e4cce816802b84cfed138dbe10a830a8b9c5630925c1e8/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~projected/kube-api-access-m7jhr DeviceMajor:0 DeviceMinor:226 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~projected/kube-api-access-2fhfx DeviceMajor:0 DeviceMinor:208 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cff1eb5a-0fc0-47ad-adff-12489a13e317/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:219 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cb6e8650291b440a345f63e5d6468ff6d9fa6bad24f03348ad4a22064df5bae7/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/77ee943e51c455b302ebb75e552facc0bdbbd74fe771a713383f94c50d9bdb22/userdata/shm DeviceMajor:0 DeviceMinor:334 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/279b261c-91e6-427e-9e19-3f1a42e60e76/volumes/kubernetes.io~projected/kube-api-access-snl4j DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77ce2b35-2046-4b1a-bb36-7f49fe33f506/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/13252367-d85f-4a6a-9eae-2e22cfbb5c4f/volumes/kubernetes.io~projected/kube-api-access-gqnln DeviceMajor:0 DeviceMinor:209 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8a21fc934d64b1daf8f99545c6e7247dcb3bb6e3fcdeec517b406bee6dd5c71/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/de24546a-df74-492e-98f2-6dcf0d466108/volumes/kubernetes.io~projected/kube-api-access-8l96t DeviceMajor:0 DeviceMinor:231 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/748b974a-4a23-4a7a-949b-3dc6f972ea61/volumes/kubernetes.io~projected/kube-api-access-rt6qt DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volumes/kubernetes.io~projected/kube-api-access-zwzn2 DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf3e1e65-7ce3-4b20-9923-98c273dd915f/volumes/kubernetes.io~projected/kube-api-access-nvhz7 DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/kube-api-access-5zjrb DeviceMajor:0 DeviceMinor:229 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a403a38582bc34a4a061361dd926dd85c83ef5e684d6394a57bba53043fd5140/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/861ecda3-10bb-451b-bf57-f62ef9cd4129/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7308f12c-06da-41a7-b2b1-ab82200985ee/volumes/kubernetes.io~projected/kube-api-access-bxbmh DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e27601fc-9881-482b-9115-a169779efa5b/volumes/kubernetes.io~projected/kube-api-access-mmgkh DeviceMajor:0 DeviceMinor:211 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e69d8d91-a4a3-412d-ab1b-d9a8409580fa/volumes/kubernetes.io~projected/kube-api-access-5wrq2 DeviceMajor:0 DeviceMinor:215 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:217 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d127029-258f-4c97-a9b8-d2cf85123890/volumes/kubernetes.io~projected/kube-api-access-2n7l9 DeviceMajor:0 DeviceMinor:214 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/154f434a774d77a1e3231b1dffc362588ebdb9ed197d92fd8de126d27551197d/userdata/shm DeviceMajor:0 DeviceMinor:221 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/310d68ba-e235-4caf-ba78-c1940e74354c/volumes/kubernetes.io~projected/kube-api-access-nx997 DeviceMajor:0 DeviceMinor:227 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:230 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc/volumes/kubernetes.io~projected/kube-api-access-cxfsz DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/666afb05-0236-4db8-8229-0871af2ff16f/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3af857443715187d65401c9b3b5941a739ac0b2307510d8b40a1212945eda39e/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fca72a21-1b7a-4005-9baa-970f86ebd652/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc3ff6e791863cf1f3b19f7a442547cb5cb39bfb43fad670da4af94fea39d52a/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e5507babc57db39013af32b88825399c14125786341fd502999ca744aa9054a1/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f89ad751c0a5eec6f927bbae97c6888f6fcba86508d4526971315bdb0ec62bc5/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:205 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7265c58c-0d6e-4fe4-98af-9db6238dd3b8/volumes/kubernetes.io~projected/kube-api-access-ff5qc DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a31ee4704c1a7f3a9b874a489ba8eb27cea9c90cc4571ce6caf8ea0a06f1f6d3/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f46a5e6f-f333-4f27-a660-2cb0bf7c3470/volumes/kubernetes.io~projected/kube-api-access-jnl2m DeviceMajor:0 DeviceMinor:210 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28ed2be66f04084c9eca971198c00b98fd3e89a525b0a5a2643f3660dfa6f62e/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5c9bcad4214aa23878ee9ef76ef82012d2ad0a40974562e9aaf30ed9ca76dafa/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0e6538b1-d3ff-4e67-81e0-e147ce5d03db/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:268 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/72fa15a84d951ae711e6008488912111d0c65596b8b915b0a1d7d531f8793ceb/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ad1dd7893496396ea9e505450a2a22e9c4869b88153f21bef4f1b0935b4c6b2b/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf0111a4-adeb-451d-a0c2-17469dce5b90/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:220 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/808b5fa2-8623-4639-870a-be16d38cafeb/volumes/kubernetes.io~projected/kube-api-access-k87cx DeviceMajor:0 DeviceMinor:216 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9621ad3348d8a349abaf3f73c3b12533a2d57fdc07e36bfbd12352e57c94e9f8/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/973842176f7c86ab869bb845de75836d326022b700709b44ee039945f79da280/userdata/shm DeviceMajor:0 DeviceMinor:241 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:218 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/19bff4949b4eb9f135f013f19c4007a014a67ee322aceb5a74264582633ab41c/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5e0946594784258e5575d4109680098a4e825037e467a2f0b421963115de8ec1/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8feb3890-bb59-4850-a7cf-7c53c9048ad1/volumes/kubernetes.io~projected/kube-api-access-xm9qd DeviceMajor:0 DeviceMinor:337 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:154f434a774d77a MacAddress:3e:96:d2:85:fd:1f Speed:10000 Mtu:8901} {Name:19bff4949b4eb9f MacAddress:e6:ac:04:ad:67:d4 Speed:10000 Mtu:8901} {Name:218f66fbe7bfb79 MacAddress:1a:fb:b2:c9:5b:12 Speed:10000 Mtu:8901} {Name:28ed2be66f04084 MacAddress:b6:00:ac:13:12:a1 Speed:10000 Mtu:8901} {Name:3af857443715187 MacAddress:c6:52:57:ba:b7:41 Speed:10000 Mtu:8901} {Name:77ee943e51c455b MacAddress:b6:5e:98:53:56:5e Speed:10000 Mtu:8901} {Name:973842176f7c86a MacAddress:e2:d2:76:92:14:b6 Speed:10000 Mtu:8901} {Name:a31ee4704c1a7f3 MacAddress:82:81:76:2d:b0:6a Speed:10000 Mtu:8901} {Name:a403a38582bc34a MacAddress:6a:f0:3e:fe:88:61 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:de:81:19:2d Speed:0 Mtu:9001} {Name:br-int MacAddress:ca:80:07:91:c4:d8 Speed:0 Mtu:8901} {Name:c8a21fc934d64b1 MacAddress:4e:01:2d:ca:49:b2 Speed:10000 Mtu:8901} {Name:cb6e8650291b440 MacAddress:a2:53:a6:b2:94:8a Speed:10000 Mtu:8901} {Name:cc3ff6e791863cf MacAddress:06:b9:67:48:46:e2 Speed:10000 Mtu:8901} {Name:e5507babc57db39 MacAddress:56:5c:3b:5d:75:8a Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:de:81:19:2d Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:26:f0:65:bf:83:22 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ca:8b:de:90:a6:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.338808    4938 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.338870    4938 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339160    4938 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339188    4938 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339348    4938 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339355    4938 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339369    4938 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339387    4938 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339407    4938 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339553    4938 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339662    4938 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339695    4938 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339704    4938 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339714    4938 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339723    4938 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.339731    4938 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.340454    4938 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.340464    4938 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:37:18.342138 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.340909    4938 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:37:18.343347 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343035    4938 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:37:18.343347 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343045    4938 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:37:18.343347 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343139    4938 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:37:18.343347 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343337    4938 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343366    4938 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343487    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343499    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343505    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343510    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343516    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:37:18.343520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343523    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343529    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343535    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343542    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343548    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343565    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343575    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343604    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:37:18.343851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.343610    4938 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:37:18.347108 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.347095    4938 server.go:1263] "Started kubelet"
Oct 30 18:37:18.347181 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.347157    4938 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:37:18.347788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.347273    4938 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:37:18.347788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.347352    4938 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:37:18.347878 ip-10-0-132-168 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:37:18.350494 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.348559    4938 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:37:18.350494 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.349228    4938 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:37:18.352045 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352027    4938 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:37:18.352143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352065    4938 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352233    4938 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:39 +0000 UTC" deadline="2028-05-01 07:35:35.831849731 +0000 UTC"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352282    4938 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352291    4938 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13164h58m17.479562991s"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352296    4938 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.352302    4938 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:37:18.353464 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.352292    4938 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-168.ec2.internal\" not found"
Oct 30 18:37:18.354846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.354769    4938 factory.go:55] Registering systemd factory
Oct 30 18:37:18.354846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.354791    4938 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:37:18.355283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355227    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="861ecda3-10bb-451b-bf57-f62ef9cd4129" volumeName="kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:37:18.355353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355291    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90889eb1-58bd-4e0a-802c-3156bbe5ef2f" volumeName="kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:37:18.355353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355307    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp" seLinuxMountContext=""
Oct 30 18:37:18.355353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355321    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf3e1e65-7ce3-4b20-9923-98c273dd915f" volumeName="kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:37:18.355353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355334    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77ce2b35-2046-4b1a-bb36-7f49fe33f506" volumeName="kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert" seLinuxMountContext=""
Oct 30 18:37:18.355353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355346    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="808b5fa2-8623-4639-870a-be16d38cafeb" volumeName="kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355358    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cca6e1a6-ea37-482c-9c61-1a7ac0259030" volumeName="kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355375    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de24546a-df74-492e-98f2-6dcf0d466108" volumeName="kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355390    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355402    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="861ecda3-10bb-451b-bf57-f62ef9cd4129" volumeName="kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355415    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fca72a21-1b7a-4005-9baa-970f86ebd652" volumeName="kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355427    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355438    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="808b5fa2-8623-4639-870a-be16d38cafeb" volumeName="kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355451    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="808b5fa2-8623-4639-870a-be16d38cafeb" volumeName="kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355466    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355478    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355491    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="310d68ba-e235-4caf-ba78-c1940e74354c" volumeName="kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355503    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7308f12c-06da-41a7-b2b1-ab82200985ee" volumeName="kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355514    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355525    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355536    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8feb3890-bb59-4850-a7cf-7c53c9048ad1" volumeName="kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355546    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16f258b8-0324-4e3a-85a1-50c4194f593d" volumeName="kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355558    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d135ba2-ec2b-4cc2-a1a9-478ba254e16f" volumeName="kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355584    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="666afb05-0236-4db8-8229-0871af2ff16f" volumeName="kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate" seLinuxMountContext=""
Oct 30 18:37:18.355583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355595    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f35ce557-816c-4b2e-97cc-19937efb59db" volumeName="kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355610    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="279b261c-91e6-427e-9e19-3f1a42e60e76" volumeName="kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355623    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7308f12c-06da-41a7-b2b1-ab82200985ee" volumeName="kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355634    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355652    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cca6e1a6-ea37-482c-9c61-1a7ac0259030" volumeName="kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355664    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="279b261c-91e6-427e-9e19-3f1a42e60e76" volumeName="kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355678    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d127029-258f-4c97-a9b8-d2cf85123890" volumeName="kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355690    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355702    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cca6e1a6-ea37-482c-9c61-1a7ac0259030" volumeName="kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355714    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf3e1e65-7ce3-4b20-9923-98c273dd915f" volumeName="kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355727    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e27601fc-9881-482b-9115-a169779efa5b" volumeName="kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355740    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f46a5e6f-f333-4f27-a660-2cb0bf7c3470" volumeName="kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355753    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13252367-d85f-4a6a-9eae-2e22cfbb5c4f" volumeName="kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355765    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc" volumeName="kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355778    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7308f12c-06da-41a7-b2b1-ab82200985ee" volumeName="kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355791    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355810    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="861ecda3-10bb-451b-bf57-f62ef9cd4129" volumeName="kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355823    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cff1eb5a-0fc0-47ad-adff-12489a13e317" volumeName="kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355838    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="808b5fa2-8623-4639-870a-be16d38cafeb" volumeName="kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355850    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355862    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77ce2b35-2046-4b1a-bb36-7f49fe33f506" volumeName="kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355873    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355886    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355899    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e27601fc-9881-482b-9115-a169779efa5b" volumeName="kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355911    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13252367-d85f-4a6a-9eae-2e22cfbb5c4f" volumeName="kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355919    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d135ba2-ec2b-4cc2-a1a9-478ba254e16f" volumeName="kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355927    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="666afb05-0236-4db8-8229-0871af2ff16f" volumeName="kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355935    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355944    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf3e1e65-7ce3-4b20-9923-98c273dd915f" volumeName="kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355951    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e69d8d91-a4a3-412d-ab1b-d9a8409580fa" volumeName="kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355959    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="666afb05-0236-4db8-8229-0871af2ff16f" volumeName="kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355967    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="861ecda3-10bb-451b-bf57-f62ef9cd4129" volumeName="kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355981    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355990    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.355999    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7265c58c-0d6e-4fe4-98af-9db6238dd3b8" volumeName="kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356007    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ba4b2b7-3196-4769-aaa9-8c40dd282893" volumeName="kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356015    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="861ecda3-10bb-451b-bf57-f62ef9cd4129" volumeName="kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356022    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf3e1e65-7ce3-4b20-9923-98c273dd915f" volumeName="kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356030    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fca72a21-1b7a-4005-9baa-970f86ebd652" volumeName="kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356037    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1d135ba2-ec2b-4cc2-a1a9-478ba254e16f" volumeName="kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356046    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493" volumeName="kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356055    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356066    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ba4b2b7-3196-4769-aaa9-8c40dd282893" volumeName="kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356074    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="90889eb1-58bd-4e0a-802c-3156bbe5ef2f" volumeName="kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356082    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cff1eb5a-0fc0-47ad-adff-12489a13e317" volumeName="kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356089    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e27601fc-9881-482b-9115-a169779efa5b" volumeName="kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356098    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="748b974a-4a23-4a7a-949b-3dc6f972ea61" volumeName="kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356106    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8feb3890-bb59-4850-a7cf-7c53c9048ad1" volumeName="kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356113    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf0111a4-adeb-451d-a0c2-17469dce5b90" volumeName="kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356130    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f46a5e6f-f333-4f27-a660-2cb0bf7c3470" volumeName="kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356140    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e6538b1-d3ff-4e67-81e0-e147ce5d03db" volumeName="kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret" seLinuxMountContext=""
Oct 30 18:37:18.356135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356152    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="50f0c5e2-b263-4047-b980-20b8ebb07a79" volumeName="kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration" seLinuxMountContext=""
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356164    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77ce2b35-2046-4b1a-bb36-7f49fe33f506" volumeName="kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config" seLinuxMountContext=""
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356177    4938 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="82214171-38b1-45d5-8c9b-794a142335fc" volumeName="kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token" seLinuxMountContext=""
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356190    4938 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356197    4938 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356222    4938 factory.go:153] Registering CRI-O factory
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356241    4938 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356334    4938 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356357    4938 factory.go:103] Registering Raw factory
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356375    4938 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:37:18.358207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.356775    4938 manager.go:319] Starting recovery of all containers
Oct 30 18:37:18.371388 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.371358    4938 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:37:18.379391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.379358    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:18.380585 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.380562    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:18.385487 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.385468    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:37:18.386709 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.386686    4938 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:37:18.386836 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.386712    4938 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:37:18.386836 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.386736    4938 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:37:18.386836 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.386788    4938 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:37:18.386836 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.386812    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:37:18.399474 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.399444    4938 generic.go:356] "Generic (PLEG): container finished" podID="22ce239b215936dbe4b6418b6fb4e9c3" containerID="3d9d04d42b9b1ecdbf0b28296bc4394fbc47b4e4e1e3e91736123d443be6918b" exitCode=0
Oct 30 18:37:18.402297 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.402278    4938 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:37:18.402398 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.402381    4938 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:37:18.403190 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.403174    4938 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:37:18.408095 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.408073    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:18.408163 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.408107    4938 generic.go:356] "Generic (PLEG): container finished" podID="808b5fa2-8623-4639-870a-be16d38cafeb" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b" exitCode=255
Oct 30 18:37:18.416188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.416133    4938 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:37:18.416355 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.416338    4938 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:37:18.433713 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433684    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="943d35aaa9714a5961e2e16712daa84d8f7be8abd8801e93263fcd5b9c17f8c4" exitCode=0
Oct 30 18:37:18.433713 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433711    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="0a99cd508fb248a31b669c8713b78e6c5cf0792331f263b64c2d013656856d94" exitCode=0
Oct 30 18:37:18.433842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433724    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="86efa1157f5a9e3a8e1fb04c3ac7bdab11434ce0d575cd0351d3f7bc977716cd" exitCode=0
Oct 30 18:37:18.433842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433733    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="afa2a2535f2483c7272d1d7a550d7000aa7a9cc8d74aa497cf29878a1fb0e9a2" exitCode=0
Oct 30 18:37:18.433842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433742    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="558de952087e59ba4596078e0970b0fd937b4f5dc58ed1fb7a1f739b4f7b2b23" exitCode=0
Oct 30 18:37:18.433842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.433754    4938 generic.go:356] "Generic (PLEG): container finished" podID="cf3e1e65-7ce3-4b20-9923-98c273dd915f" containerID="e5f3a5b8889ec77294e20b56374af895423fed117de18dba110dde8695aafe36" exitCode=0
Oct 30 18:37:18.443686 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.443664    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:37:18.444047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.444023    4938 generic.go:356] "Generic (PLEG): container finished" podID="861ecda3-10bb-451b-bf57-f62ef9cd4129" containerID="5bcafd92b13acaa4c5a973b2648903b52b2e79a8d903a4a8540aaf7f0bbb73da" exitCode=1
Oct 30 18:37:18.451765 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.451750    4938 manager.go:324] Recovery completed
Oct 30 18:37:18.469959 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.469935    4938 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:37:18.469959 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.469958    4938 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:37:18.470090 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.469979    4938 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:37:18.470172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470148    4938 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:37:18.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470175    4938 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:37:18.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470200    4938 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:37:18.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470208    4938 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:37:18.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470215    4938 policy_none.go:50] "Start"
Oct 30 18:37:18.470408 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470233    4938 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:37:18.470408 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470262    4938 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:37:18.470499 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470484    4938 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:37:18.470499 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470495    4938 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:37:18.470592 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.470503    4938 policy_none.go:44] "Start"
Oct 30 18:37:18.475079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.475053    4938 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:37:18.477612 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.477600    4938 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:37:18.478061 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.477623    4938 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:37:18.478061 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478061    4938 server.go:87] "Starting device plugin registration server"
Oct 30 18:37:18.478327 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478317    4938 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:37:18.478369 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478328    4938 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:37:18.478456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478440    4938 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:37:18.478575 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478543    4938 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:37:18.478634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478587    4938 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:37:18.478634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478594    4938 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:37:18.478729 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.478651    4938 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:37:18.478574605Z","UUID":"1ef905b7-0973-4f36-8de0-fdc985d73078","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:37:18.479929 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.479911    4938 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:37:18.480053 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.480038    4938 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:37:18.480111 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.480068    4938 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:37:18.486894 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.486857    4938 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"]
Oct 30 18:37:18.487592 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487573    4938 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.487689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487579    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerStarted","Data":"72542513d6f66e7885c9b3827c307c665b2baf03311a5cdd48f0e7cddc5ed536"}
Oct 30 18:37:18.487689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487625    4938 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.487689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487635    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerDied","Data":"3d9d04d42b9b1ecdbf0b28296bc4394fbc47b4e4e1e3e91736123d443be6918b"}
Oct 30 18:37:18.487689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487661    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal" event={"ID":"22ce239b215936dbe4b6418b6fb4e9c3","Type":"ContainerStarted","Data":"eb09b327c8b2f2cb5cabbd90071aad9135707207ba0e59bf49cd007e9121570d"}
Oct 30 18:37:18.487877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487727    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"d35ca846eedff7c0a375d6f9a1bb598c","Type":"ContainerStarted","Data":"6f2c5d56636029043503c347efa50791e7dfe02bf598b945e09b2b7b552b66bb"}
Oct 30 18:37:18.487877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.487748    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal" event={"ID":"d35ca846eedff7c0a375d6f9a1bb598c","Type":"ContainerStarted","Data":"e0d837ea561ab5f54de545466e7c38a34e138d60f1295ac0e64f2319988d6d3f"}
Oct 30 18:37:18.509065 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.509040    4938 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:37:18.509136 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.509097    4938 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.509826 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.509812    4938 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:37:18.509873 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:18.509844    4938 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.579283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.579239    4938 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:37:18.580547 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.580530    4938 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 18:37:18.580631 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.580560    4938 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 18:37:18.580631 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.580570    4938 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-168.ec2.internal" event="NodeHasSufficientPID"
Oct 30 18:37:18.580631 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.580606    4938 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.602738 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.602714    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.602814 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.602744    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.602814 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.602762    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.615545 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.615523    4938 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.615609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.615584    4938 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703368 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703341    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703379    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703405    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703441    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703445    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d35ca846eedff7c0a375d6f9a1bb598c-config\") pod \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" (UID: \"d35ca846eedff7c0a375d6f9a1bb598c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:18.703462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:18.703452    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22ce239b215936dbe4b6418b6fb4e9c3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" (UID: \"22ce239b215936dbe4b6418b6fb4e9c3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:19.340486 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.340452    4938 apiserver.go:52] "Watching apiserver"
Oct 30 18:37:19.409329 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.409302    4938 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:37:19.409963 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.409941    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-ghjt5","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt","openshift-multus/multus-additional-cni-plugins-n9ff9","openshift-console-operator/console-operator-7cd56b8dbc-rzcch","openshift-dns/node-resolver-j4266","openshift-image-registry/image-registry-8457959978-p9454","openshift-ingress/router-default-c6947b789-btt56","openshift-insights/insights-operator-7c4986d449-mvhgb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2","openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd","openshift-image-registry/image-registry-5bbf88fc59-82jt6","openshift-network-diagnostics/network-check-source-6f78d59f99-49spc","openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg","openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9","kube-system/konnectivity-agent-6bz4h","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh","openshift-insights/insights-runtime-extractor-p9vkt","kube-system/global-pull-secret-syncer-kp5w5","openshift-cluster-node-tuning-operator/tuned-rwlfm","kube-system/kas-connection-checker-685895c7c9-96hgm","kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal","openshift-ingress-canary/ingress-canary-tkz7q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal","openshift-multus/network-metrics-daemon-mlgx9","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8","openshift-ovn-kubernetes/ovnkube-node-k5ctz","openshift-network-console/networking-console-plugin-86c478bb88-r6r8p","openshift-dns/dns-default-6w2g6","kube-system/kas-connection-checker-685895c7c9-zh56w","kube-system/kas-connection-checker-685895c7c9-5kbfh","openshift-network-diagnostics/network-check-target-kps9t","openshift-network-operator/iptables-alerter-8tlpv","openshift-multus/multus-l9qs4"]
Oct 30 18:37:19.411632 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.411605    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.412267 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.412237    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.413018 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.413000    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.414196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.414181    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:19.414713 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.414694    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.419860 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.419845    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:19.425873 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.425853    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.426359 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.426344    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.427857 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.427833    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 30 18:37:19.427973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.427869    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 30 18:37:19.427973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.427883    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 30 18:37:19.427973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.427914    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.428134 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.428104    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-g4ls8\""
Oct 30 18:37:19.428134 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.428131    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7rwzw\""
Oct 30 18:37:19.428242 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.428140    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-nczkq\""
Oct 30 18:37:19.444664 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.444644    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 30 18:37:19.445703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.445684    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.445703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.445687    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.445839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.445714    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.445839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.445714    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 30 18:37:19.445839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.445735    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 30 18:37:19.446213 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446194    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.446370 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446353    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.446411 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446362    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.446455 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446411    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.446514 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446468    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 30 18:37:19.446514 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.446475    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.447174 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447154    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.447174 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447166    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.447428 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447274    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.447428 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447330    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 30 18:37:19.447428 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447385    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.447428 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447397    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.447721 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447447    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 30 18:37:19.447721 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447612    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.447911 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447890    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.447965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447928    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.448013 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.447998    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.448067 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.448054    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.448370 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.448358    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.451339 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.451323    4938 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:19.451458 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.451446    4938 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:19.454158 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.454138    4938 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:37:19.460839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460821    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.460922 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460845    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.460922 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460911    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461049 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460952    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.461049 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460968    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.461049 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460991    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.461049 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461028    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460978    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461064    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461033    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461045    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.460973    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461063    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461074    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461131    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461195    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 18:37:19.461228 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461214    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461267    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461316    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461344    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461384    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461407    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461412    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461440    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461508    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461513    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461561    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461592    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461613    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.461648 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461645    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461660    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461693    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461725    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461692    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461751    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:37:19.462058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.461776    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:37:19.462327 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.462313    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 18:37:19.470338 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.470311    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:37:19.470465 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.470441    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 30 18:37:19.470577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.470492    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 18:37:19.470577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.470546    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:37:19.471395 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.471373    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 30 18:37:19.471555 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.471523    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 30 18:37:19.471824 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.471802    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bx2pm\""
Oct 30 18:37:19.472021 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472002    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:37:19.472322 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472305    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-t7f6r\""
Oct 30 18:37:19.472564 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472548    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:37:19.472633 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472613    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h68zm\""
Oct 30 18:37:19.472763 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472744    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6dtmn\""
Oct 30 18:37:19.472871 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472761    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7kr8g\""
Oct 30 18:37:19.472927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472892    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d48sc\""
Oct 30 18:37:19.472927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472915    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9ktdg\""
Oct 30 18:37:19.473027 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.472743    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:37:19.473073 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.473048    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:37:19.473073 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.473065    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vjc5x\""
Oct 30 18:37:19.474562 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474541    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 30 18:37:19.474562 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474546    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b8l82\""
Oct 30 18:37:19.474758 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474727    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 30 18:37:19.474758 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474733    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r66x4\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474792    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t7qvq\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474818    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-d56w6\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474835    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gwgkn\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474852    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dvz9x\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474885    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dzmp9\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474894    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gtrxb\""
Oct 30 18:37:19.474935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474916    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474963    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zcgpq\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.474995    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475022    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-rbclc\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475061    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5w9ll\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475077    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-64xr6\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475068    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-84qrk\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475110    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475148    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vr4rx\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475194    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475228    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mxkzp\""
Oct 30 18:37:19.475290 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475281    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:37:19.475710 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475526    4938 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:37:19.475710 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.475588    4938 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:19.475933 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.475917    4938 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:37:19.476023 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.475959    4938 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-168.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-168.ec2.internal"
Oct 30 18:37:19.476808 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.476792    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-mdkqz\""
Oct 30 18:37:19.504062 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.504039    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:19.507079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507054    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.507154 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507086    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.507154 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507103    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n7l9\" (UniqueName: \"kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9\") pod \"kas-connection-checker-685895c7c9-96hgm\" (UID: \"4d127029-258f-4c97-a9b8-d2cf85123890\") " pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:19.507154 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507120    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.507282 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507200    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.507282 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507268    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.507354 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507292    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wwgx\" (UniqueName: \"kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:19.507432 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507412    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.507467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507449    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5zx\" (UniqueName: \"kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.507501 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507476    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.507530 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507501    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.507559 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507532    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wrq2\" (UniqueName: \"kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2\") pod \"kas-connection-checker-685895c7c9-5kbfh\" (UID: \"e69d8d91-a4a3-412d-ab1b-d9a8409580fa\") " pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:19.507590 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507571    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.507622 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507607    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.507654 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507641    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.507696 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507681    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:37:19.507730 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507716    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.507760 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507740    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.507794 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507757    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k87cx\" (UniqueName: \"kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.507794 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507779    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.507876 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507837    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fhfx\" (UniqueName: \"kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.507876 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507866    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.507965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507895    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.507965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507919    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fca72a21-1b7a-4005-9baa-970f86ebd652-konnectivity-ca\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:37:19.507965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507923    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bf0111a4-adeb-451d-a0c2-17469dce5b90-serving-cert\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.507965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507935    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.507965 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507923    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.507991    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508027    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508048    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvhz7\" (UniqueName: \"kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508063    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508080    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508100    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ba4b2b7-3196-4769-aaa9-8c40dd282893-tmp-dir\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508110    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508135    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508188    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.508237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508222    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508273    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e27601fc-9881-482b-9115-a169779efa5b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508289    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508374    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfppz\" (UniqueName: \"kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508410    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-env-overrides\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508412    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508455    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508501    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovn-node-metrics-cert\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508565    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508607    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/77ce2b35-2046-4b1a-bb36-7f49fe33f506-config\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508610    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508660    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.508701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508690    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508723    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508730    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnl2m\" (UniqueName: \"kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508759    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-daemon-config\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508780    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508916    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508952    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.508993    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jhr\" (UniqueName: \"kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509088    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77ce2b35-2046-4b1a-bb36-7f49fe33f506-serving-cert\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509101    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509130    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm9qd\" (UniqueName: \"kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.509199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509187    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509211    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509238    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509311    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt6qt\" (UniqueName: \"kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509450    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509455    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e27601fc-9881-482b-9115-a169779efa5b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509481    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509510    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509530    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509554    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509569    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509604    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.509638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509636    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509663    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509658    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-snapshots\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509676    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-config\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509742    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509746    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-service-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509759    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509795    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509814    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509828    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509861    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwzn2\" (UniqueName: \"kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509882    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509897    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509926    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.509981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509969    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.509993    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510028    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510047    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/808b5fa2-8623-4639-870a-be16d38cafeb-serving-cert\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510117    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510174    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f35ce557-816c-4b2e-97cc-19937efb59db-nginx-conf\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510199    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510283    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.510521 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510285    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.510757 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510716    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.510757 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510750    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.510817 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510752    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.510817 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510786    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxfsz\" (UniqueName: \"kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:19.510904 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510891    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.510939 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510915    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.510939 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510933    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.510999 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.510977    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx997\" (UniqueName: \"kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997\") pod \"volume-data-source-validator-58c9f5f46d-q4xsh\" (UID: \"310d68ba-e235-4caf-ba78-c1940e74354c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:19.511119 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511100    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.511156 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511135    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-default-certificate\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.511188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511161    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.511239 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511222    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511300 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511278    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511331 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511309    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.511362 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511327    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.511362 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511355    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.511451 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511439    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511482 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511476    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ng5d\" (UniqueName: \"kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.511514 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511495    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.511546 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511511    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cni-binary-copy\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.511546 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511513    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/861ecda3-10bb-451b-bf57-f62ef9cd4129-ovnkube-script-lib\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511604 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511567    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l96t\" (UniqueName: \"kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t\") pod \"network-check-source-6f78d59f99-49spc\" (UID: \"de24546a-df74-492e-98f2-6dcf0d466108\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:19.511640 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511610    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.511750 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511733    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.511812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511765    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.511812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511770    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-original-pull-secret\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.511812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511796    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:19.511812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511799    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.511973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511834    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.511973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511860    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511889    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.511973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511930    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.511973 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511961    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.511989    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512008    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-stats-auth\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512015    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512051    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2rd4\" (UniqueName: \"kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4\") pod \"migrator-66df5d496b-wfrjd\" (UID: \"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512073    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512095    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512120    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.512209 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512162    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512208    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512302    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff5qc\" (UniqueName: \"kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc\") pod \"kas-connection-checker-685895c7c9-zh56w\" (UID: \"7265c58c-0d6e-4fe4-98af-9db6238dd3b8\") " pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512369    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512387    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-trusted-ca\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512393    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512391    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cff1eb5a-0fc0-47ad-adff-12489a13e317-cert\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512416    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512441    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512448    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8feb3890-bb59-4850-a7cf-7c53c9048ad1-data-volume\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512449    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snl4j\" (UniqueName: \"kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512510    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512539    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512564    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf0111a4-adeb-451d-a0c2-17469dce5b90-tmp\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512568    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmgkh\" (UniqueName: \"kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.512608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512604    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512675    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512721    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512765    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512836    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-tmp-dir\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512851    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512874    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/808b5fa2-8623-4639-870a-be16d38cafeb-config\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512881    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512906    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbmh\" (UniqueName: \"kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512922    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512937    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512953    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.512972    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-tmp\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513038    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-tuned\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513055 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513059    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513074    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqnln\" (UniqueName: \"kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513104    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513135    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-config-volume\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513136    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513198    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fca72a21-1b7a-4005-9baa-970f86ebd652-agent-certs\") pod \"konnectivity-agent-6bz4h\" (UID: \"fca72a21-1b7a-4005-9baa-970f86ebd652\") " pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513212    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513277    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56cnn\" (UniqueName: \"kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513299    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513314    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513328    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513346    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513370    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjw94\" (UniqueName: \"kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513400    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513428    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513467    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513502    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513524    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf0111a4-adeb-451d-a0c2-17469dce5b90-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513531    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513554    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513579    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.513620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513603    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cni-binary-copy\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513716    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513811    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513829    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-iptables-alerter-script\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513815    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.513977    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/279b261c-91e6-427e-9e19-3f1a42e60e76-serviceca\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.514422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.514064    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.558298 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.558234    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt6qt\" (UniqueName: \"kubernetes.io/projected/748b974a-4a23-4a7a-949b-3dc6f972ea61-kube-api-access-rt6qt\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.558377 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.558298    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swtrr\" (UniqueName: \"kubernetes.io/projected/16f258b8-0324-4e3a-85a1-50c4194f593d-kube-api-access-swtrr\") pod \"network-check-target-kps9t\" (UID: \"16f258b8-0324-4e3a-85a1-50c4194f593d\") " pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:19.558835 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.558817    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfppz\" (UniqueName: \"kubernetes.io/projected/666afb05-0236-4db8-8229-0871af2ff16f-kube-api-access-vfppz\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.559237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.559214    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmgkh\" (UniqueName: \"kubernetes.io/projected/e27601fc-9881-482b-9115-a169779efa5b-kube-api-access-mmgkh\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8\" (UID: \"e27601fc-9881-482b-9115-a169779efa5b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8"
Oct 30 18:37:19.559677 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.559663    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5zx\" (UniqueName: \"kubernetes.io/projected/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-kube-api-access-rw5zx\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.560170 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.560153    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.562003 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.561973    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wrq2\" (UniqueName: \"kubernetes.io/projected/e69d8d91-a4a3-412d-ab1b-d9a8409580fa-kube-api-access-5wrq2\") pod \"kas-connection-checker-685895c7c9-5kbfh\" (UID: \"e69d8d91-a4a3-412d-ab1b-d9a8409580fa\") " pod="kube-system/kas-connection-checker-685895c7c9-5kbfh"
Oct 30 18:37:19.562456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.562437    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbmh\" (UniqueName: \"kubernetes.io/projected/7308f12c-06da-41a7-b2b1-ab82200985ee-kube-api-access-bxbmh\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.562737 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.562718    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jhr\" (UniqueName: \"kubernetes.io/projected/bf0111a4-adeb-451d-a0c2-17469dce5b90-kube-api-access-m7jhr\") pod \"insights-operator-7c4986d449-mvhgb\" (UID: \"bf0111a4-adeb-451d-a0c2-17469dce5b90\") " pod="openshift-insights/insights-operator-7c4986d449-mvhgb"
Oct 30 18:37:19.563952 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.563932    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k87cx\" (UniqueName: \"kubernetes.io/projected/808b5fa2-8623-4639-870a-be16d38cafeb-kube-api-access-k87cx\") pod \"console-operator-7cd56b8dbc-rzcch\" (UID: \"808b5fa2-8623-4639-870a-be16d38cafeb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:19.564302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.564284    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56cnn\" (UniqueName: \"kubernetes.io/projected/cca6e1a6-ea37-482c-9c61-1a7ac0259030-kube-api-access-56cnn\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.564667 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.564648    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n7l9\" (UniqueName: \"kubernetes.io/projected/4d127029-258f-4c97-a9b8-d2cf85123890-kube-api-access-2n7l9\") pod \"kas-connection-checker-685895c7c9-96hgm\" (UID: \"4d127029-258f-4c97-a9b8-d2cf85123890\") " pod="kube-system/kas-connection-checker-685895c7c9-96hgm"
Oct 30 18:37:19.567958 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.567938    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxfsz\" (UniqueName: \"kubernetes.io/projected/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-kube-api-access-cxfsz\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:19.568061 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.568047    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l96t\" (UniqueName: \"kubernetes.io/projected/de24546a-df74-492e-98f2-6dcf0d466108-kube-api-access-8l96t\") pod \"network-check-source-6f78d59f99-49spc\" (UID: \"de24546a-df74-492e-98f2-6dcf0d466108\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-49spc"
Oct 30 18:37:19.570048 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.570032    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjw94\" (UniqueName: \"kubernetes.io/projected/7ba4b2b7-3196-4769-aaa9-8c40dd282893-kube-api-access-vjw94\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.571825 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.571809    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fhfx\" (UniqueName: \"kubernetes.io/projected/77ce2b35-2046-4b1a-bb36-7f49fe33f506-kube-api-access-2fhfx\") pod \"service-ca-operator-544d749c8c-psxz9\" (UID: \"77ce2b35-2046-4b1a-bb36-7f49fe33f506\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9"
Oct 30 18:37:19.572031 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.572015    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wwgx\" (UniqueName: \"kubernetes.io/projected/cff1eb5a-0fc0-47ad-adff-12489a13e317-kube-api-access-5wwgx\") pod \"ingress-canary-tkz7q\" (UID: \"cff1eb5a-0fc0-47ad-adff-12489a13e317\") " pod="openshift-ingress-canary/ingress-canary-tkz7q"
Oct 30 18:37:19.573353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.573332    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snl4j\" (UniqueName: \"kubernetes.io/projected/279b261c-91e6-427e-9e19-3f1a42e60e76-kube-api-access-snl4j\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.573483 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.573462    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwzn2\" (UniqueName: \"kubernetes.io/projected/861ecda3-10bb-451b-bf57-f62ef9cd4129-kube-api-access-zwzn2\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.573653 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.573640    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqnln\" (UniqueName: \"kubernetes.io/projected/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-kube-api-access-gqnln\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.574858 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.574833    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm9qd\" (UniqueName: \"kubernetes.io/projected/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-api-access-xm9qd\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.574948 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.574866    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff5qc\" (UniqueName: \"kubernetes.io/projected/7265c58c-0d6e-4fe4-98af-9db6238dd3b8-kube-api-access-ff5qc\") pod \"kas-connection-checker-685895c7c9-zh56w\" (UID: \"7265c58c-0d6e-4fe4-98af-9db6238dd3b8\") " pod="kube-system/kas-connection-checker-685895c7c9-zh56w"
Oct 30 18:37:19.575301 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.575283    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx997\" (UniqueName: \"kubernetes.io/projected/310d68ba-e235-4caf-ba78-c1940e74354c-kube-api-access-nx997\") pod \"volume-data-source-validator-58c9f5f46d-q4xsh\" (UID: \"310d68ba-e235-4caf-ba78-c1940e74354c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-q4xsh"
Oct 30 18:37:19.575372 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.575315    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvhz7\" (UniqueName: \"kubernetes.io/projected/cf3e1e65-7ce3-4b20-9923-98c273dd915f-kube-api-access-nvhz7\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.575444 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.575430    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnl2m\" (UniqueName: \"kubernetes.io/projected/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-kube-api-access-jnl2m\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.575477 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.575438    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ng5d\" (UniqueName: \"kubernetes.io/projected/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-kube-api-access-2ng5d\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.575636 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.575619    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2rd4\" (UniqueName: \"kubernetes.io/projected/1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493-kube-api-access-c2rd4\") pod \"migrator-66df5d496b-wfrjd\" (UID: \"1ede0bf6-2fb1-49f4-85d3-eb6f82b5d493\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-wfrjd"
Oct 30 18:37:19.576993 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.576975    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.578495 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.578477    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.581802 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.581786    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.614500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614449    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.614500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614481    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.614500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614497    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614522    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614545    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614569    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614574    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ba4b2b7-3196-4769-aaa9-8c40dd282893-hosts-file\") pod \"node-resolver-j4266\" (UID: \"7ba4b2b7-3196-4769-aaa9-8c40dd282893\") " pod="openshift-dns/node-resolver-j4266"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614584    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-netns\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614591    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614634    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614634    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-run-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.614643    4938 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614663    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614687    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-socket-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614697    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-conf\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.614703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614704    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614697    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-multus\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.614751    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.114730274 +0000 UTC m=+1.874738735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614766    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614813    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614814    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-modprobe-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614858    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8feb3890-bb59-4850-a7cf-7c53c9048ad1-crio-socket\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614883    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysctl-d\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614891    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614895    4938 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614928    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-sys\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614928    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614958    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614967    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-kubernetes\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.614979    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615007    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-k8s-cni-cncf-io\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615005    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615038    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615043    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-device-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615054    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615076    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-etc-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615045    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-lib-modules\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615077    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615103    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615121    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-dbus\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615126    4938 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615123    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615161    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-run\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615166    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-system-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615185    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.115164062 +0000 UTC m=+1.875172502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615229    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615272    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.615271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615293    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615309    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615332    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615352    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615361    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615362    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-socket-dir-parent\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615393    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615401    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615413    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-etc-selinux\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615419    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615441    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-netns\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615452    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-log-socket\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615470    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615497    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615524    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615558    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615590    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0e6538b1-d3ff-4e67-81e0-e147ce5d03db-kubelet-config\") pod \"global-pull-secret-syncer-kp5w5\" (UID: \"0e6538b1-d3ff-4e67-81e0-e147ce5d03db\") " pod="kube-system/global-pull-secret-syncer-kp5w5"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615599    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615630    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615650    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-hostroot\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615659    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615684    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615716    4938 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615749    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615755    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-ovn\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615717    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-netd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615778    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.115762311 +0000 UTC m=+1.875770759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : secret "metrics-daemon-secret" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615633    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-kubelet\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615711    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-system-cni-dir\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615791    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-kubelet\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615814    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615839    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615841    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-sys-fs\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615869    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615880    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-systemd-units\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615893    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8feb3890-bb59-4850-a7cf-7c53c9048ad1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615916    4938 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615897    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615933    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-systemd\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615915    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-slash\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.615955    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.115945078 +0000 UTC m=+1.875953524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615968    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615995    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.615995    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/90889eb1-58bd-4e0a-802c-3156bbe5ef2f-host-slash\") pod \"iptables-alerter-8tlpv\" (UID: \"90889eb1-58bd-4e0a-802c-3156bbe5ef2f\") " pod="openshift-network-operator/iptables-alerter-8tlpv"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616024    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-run-systemd\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616032    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616013    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-cni-bin\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616068    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616095    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616122    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616146    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616150    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-run-multus-certs\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616156    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-etc-kubernetes\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616231    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616277    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616288    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-cnibin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616303    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616329    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616332    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-var-lib-kubelet\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616365    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cf3e1e65-7ce3-4b20-9923-98c273dd915f-cnibin\") pod \"multus-additional-cni-plugins-n9ff9\" (UID: \"cf3e1e65-7ce3-4b20-9923-98c273dd915f\") " pod="openshift-multus/multus-additional-cni-plugins-n9ff9"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616375    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-host\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616385    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616423    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616451    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616483    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616514    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.616538    4938 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616543    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616570    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.616598    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls podName:13252367-d85f-4a6a-9eae-2e22cfbb5c4f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.116582122 +0000 UTC m=+1.876590577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-csnrg" (UID: "13252367-d85f-4a6a-9eae-2e22cfbb5c4f") : secret "cluster-monitoring-operator-tls" not found
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616619    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616635    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7308f12c-06da-41a7-b2b1-ab82200985ee-etc-sysconfig\") pod \"tuned-rwlfm\" (UID: \"7308f12c-06da-41a7-b2b1-ab82200985ee\") " pod="openshift-cluster-node-tuning-operator/tuned-rwlfm"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616649    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616687    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-node-log\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616705    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616706    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-conf-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616730    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616738    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616652    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-host-var-lib-cni-bin\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616778    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/279b261c-91e6-427e-9e19-3f1a42e60e76-host\") pod \"node-ca-ghjt5\" (UID: \"279b261c-91e6-427e-9e19-3f1a42e60e76\") " pod="openshift-image-registry/node-ca-ghjt5"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616826    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cca6e1a6-ea37-482c-9c61-1a7ac0259030-multus-cni-dir\") pod \"multus-l9qs4\" (UID: \"cca6e1a6-ea37-482c-9c61-1a7ac0259030\") " pod="openshift-multus/multus-l9qs4"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616828    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/861ecda3-10bb-451b-bf57-f62ef9cd4129-var-lib-openvswitch\") pod \"ovnkube-node-k5ctz\" (UID: \"861ecda3-10bb-451b-bf57-f62ef9cd4129\") " pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:19.616792 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:19.616849    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:20.11682134 +0000 UTC m=+1.876829796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616871    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.616905    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/748b974a-4a23-4a7a-949b-3dc6f972ea61-registration-dir\") pod \"aws-ebs-csi-driver-node-h4jc2\" (UID: \"748b974a-4a23-4a7a-949b-3dc6f972ea61\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h4jc2"
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.617617    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f46a5e6f-f333-4f27-a660-2cb0bf7c3470-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-klrgt\" (UID: \"f46a5e6f-f333-4f27-a660-2cb0bf7c3470\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.617839    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/666afb05-0236-4db8-8229-0871af2ff16f-metrics-certs\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.617867    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"image-registry-8457959978-p9454\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") " pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.619609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.618645    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"image-registry-5bbf88fc59-82jt6\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") " pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.652778 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.652754    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:19.712275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.712235    4938 scope.go:122] "RemoveContainer" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"
Oct 30 18:37:19.713081 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.712874    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:19.713132 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.713078    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:19.713236 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.713220    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"
Oct 30 18:37:19.726596 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.726578    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:19.792572 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:19.791185    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:20.059542 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.059508    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:20.062559 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:20.062521    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82214171_38b1_45d5_8c9b_794a142335fc.slice/crio-4a3ec049f55646adb44724850db3a64e4a288d7df7482cf70a3eab7b05333d50 WatchSource:0}: Error finding container 4a3ec049f55646adb44724850db3a64e4a288d7df7482cf70a3eab7b05333d50: Status 404 returned error can't find the container with id 4a3ec049f55646adb44724850db3a64e4a288d7df7482cf70a3eab7b05333d50
Oct 30 18:37:20.086895 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.086731    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:20.088893 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.088864    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt"]
Oct 30 18:37:20.124115 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124063    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:37:20.124694 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124673    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:20.124778 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124724    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:20.124778 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124762    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:20.124889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124819    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:20.124889 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124830    4938 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 18:37:20.124889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124860    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124892    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls podName:1d135ba2-ec2b-4cc2-a1a9-478ba254e16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.124873834 +0000 UTC m=+2.884882276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls") pod "dns-default-6w2g6" (UID: "1d135ba2-ec2b-4cc2-a1a9-478ba254e16f") : secret "dns-default-metrics-tls" not found
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124910    4938 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.124947    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124957    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.124942846 +0000 UTC m=+2.884951291 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124892    4938 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.124983    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.124971928 +0000 UTC m=+2.884980369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.125008    4938 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:20.125029 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.125022    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.125003918 +0000 UTC m=+2.885012358 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : secret "metrics-daemon-secret" not found
Oct 30 18:37:20.125510 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.125045    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:21.12503366 +0000 UTC m=+2.885042115 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:20.127044 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.127018    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/13252367-d85f-4a6a-9eae-2e22cfbb5c4f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-csnrg\" (UID: \"13252367-d85f-4a6a-9eae-2e22cfbb5c4f\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:20.127204 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:20.127183    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f0c5e2_b263_4047_b980_20b8ebb07a79.slice/crio-9dc9e68ad5a93261b430e11935855803580c8c8e14bbc22fc435b4c4952bc3d4 WatchSource:0}: Error finding container 9dc9e68ad5a93261b430e11935855803580c8c8e14bbc22fc435b4c4952bc3d4: Status 404 returned error can't find the container with id 9dc9e68ad5a93261b430e11935855803580c8c8e14bbc22fc435b4c4952bc3d4
Oct 30 18:37:20.156858 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.156840    4938 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:37:20.312765 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.312734    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"
Oct 30 18:37:20.382729 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.382657    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:20.457148 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.457121    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:37:20.457664 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.457644    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/1.log"
Oct 30 18:37:20.457716 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.457689    4938 generic.go:356] "Generic (PLEG): container finished" podID="808b5fa2-8623-4639-870a-be16d38cafeb" containerID="a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56" exitCode=255
Oct 30 18:37:20.457824 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.457780    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" event={"ID":"808b5fa2-8623-4639-870a-be16d38cafeb","Type":"ContainerDied","Data":"a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56"}
Oct 30 18:37:20.457951 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.457873    4938 scope.go:122] "RemoveContainer" containerID="7c473f4106ff343dd478782d0024bf53399fe1851dadb83d1efb67132898771b"
Oct 30 18:37:20.458270 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.458238    4938 scope.go:122] "RemoveContainer" containerID="a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56"
Oct 30 18:37:20.458505 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:20.458481    4938 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:20.459773 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.459736    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" event={"ID":"50f0c5e2-b263-4047-b980-20b8ebb07a79","Type":"ContainerStarted","Data":"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"}
Oct 30 18:37:20.459860 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.459801    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" event={"ID":"50f0c5e2-b263-4047-b980-20b8ebb07a79","Type":"ContainerStarted","Data":"9dc9e68ad5a93261b430e11935855803580c8c8e14bbc22fc435b4c4952bc3d4"}
Oct 30 18:37:20.461019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.460994    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt" event={"ID":"f46a5e6f-f333-4f27-a660-2cb0bf7c3470","Type":"ContainerStarted","Data":"5478213fdfd75b77300625e406229dce72d4b1d161c31358b66266fd5e7136df"}
Oct 30 18:37:20.462452 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.462421    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8457959978-p9454" event={"ID":"82214171-38b1-45d5-8c9b-794a142335fc","Type":"ContainerStarted","Data":"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"}
Oct 30 18:37:20.462542 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.462473    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8457959978-p9454" event={"ID":"82214171-38b1-45d5-8c9b-794a142335fc","Type":"ContainerStarted","Data":"4a3ec049f55646adb44724850db3a64e4a288d7df7482cf70a3eab7b05333d50"}
Oct 30 18:37:20.589345 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.589316    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4266_7ba4b2b7-3196-4769-aaa9-8c40dd282893/dns-node-resolver/0.log"
Oct 30 18:37:20.621087 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.621054    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg"]
Oct 30 18:37:20.626374 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:20.626345    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13252367_d85f_4a6a_9eae_2e22cfbb5c4f.slice/crio-752ad29c28732a7fbbaa623f142564bab3d5f0ba34d56a53131e63f71d0c0dd2 WatchSource:0}: Error finding container 752ad29c28732a7fbbaa623f142564bab3d5f0ba34d56a53131e63f71d0c0dd2: Status 404 returned error can't find the container with id 752ad29c28732a7fbbaa623f142564bab3d5f0ba34d56a53131e63f71d0c0dd2
Oct 30 18:37:20.980953 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.980907    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:20.987802 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:20.987779    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5bbf88fc59-82jt6_50f0c5e2-b263-4047-b980-20b8ebb07a79/registry/0.log"
Oct 30 18:37:21.133417 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.133375    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:21.133573 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.133435    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:21.133573 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.133487    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:21.133573 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.133512    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133610    4938 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133617    4938 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.133662    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133680    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.133662524 +0000 UTC m=+4.893670969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133617    4938 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133716    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert podName:f35ce557-816c-4b2e-97cc-19937efb59db nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.133697533 +0000 UTC m=+4.893705987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-r6r8p" (UID: "f35ce557-816c-4b2e-97cc-19937efb59db") : secret "networking-console-plugin-cert" not found
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133742    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.133732436 +0000 UTC m=+4.893740908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:21.133758 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:21.133764    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:23.133755389 +0000 UTC m=+4.893763836 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : secret "metrics-daemon-secret" not found
Oct 30 18:37:21.135915 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.135889    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1d135ba2-ec2b-4cc2-a1a9-478ba254e16f-metrics-tls\") pod \"dns-default-6w2g6\" (UID: \"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f\") " pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:21.167025 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.166991    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.189424 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.189402    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8457959978-p9454_82214171-38b1-45d5-8c9b-794a142335fc/registry/0.log"
Oct 30 18:37:21.226786 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.226760    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:21.391072 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.391032    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ghjt5_279b261c-91e6-427e-9e19-3f1a42e60e76/node-ca/0.log"
Oct 30 18:37:21.468318 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.468279    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg" event={"ID":"13252367-d85f-4a6a-9eae-2e22cfbb5c4f","Type":"ContainerStarted","Data":"752ad29c28732a7fbbaa623f142564bab3d5f0ba34d56a53131e63f71d0c0dd2"}
Oct 30 18:37:21.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.470105    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:37:21.470224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.470206    4938 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:37:21.536481 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.536411    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6w2g6"]
Oct 30 18:37:21.540438 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:21.540405    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d135ba2_ec2b_4cc2_a1a9_478ba254e16f.slice/crio-8d2e9b6d34e09f96745f7c716570fa89e32747f41ab00ed79969e451a6b5a128 WatchSource:0}: Error finding container 8d2e9b6d34e09f96745f7c716570fa89e32747f41ab00ed79969e451a6b5a128: Status 404 returned error can't find the container with id 8d2e9b6d34e09f96745f7c716570fa89e32747f41ab00ed79969e451a6b5a128
Oct 30 18:37:21.571169 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.571145    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:21.786784 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:21.785087    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:22.366223 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.366194    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:22.475008 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.474976    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6w2g6" event={"ID":"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f","Type":"ContainerStarted","Data":"8d2e9b6d34e09f96745f7c716570fa89e32747f41ab00ed79969e451a6b5a128"}
Oct 30 18:37:22.476797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.476757    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-csnrg" event={"ID":"13252367-d85f-4a6a-9eae-2e22cfbb5c4f","Type":"ContainerStarted","Data":"8733142db6bc7720d7609b27176d9aa706bbab62ab4a9270bce8046b511db954"}
Oct 30 18:37:22.478789 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.478764    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt" event={"ID":"f46a5e6f-f333-4f27-a660-2cb0bf7c3470","Type":"ContainerStarted","Data":"1022cba89ee810596bfcc45314e6facbe7b6505e6057f99d3fbd4231dea6c8b3"}
Oct 30 18:37:22.478906 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.478812    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-klrgt" event={"ID":"f46a5e6f-f333-4f27-a660-2cb0bf7c3470","Type":"ContainerStarted","Data":"1ab30edce2d2db4a2642862f5a6ad620361cf3a9f040d47725c1b5bc7b74cc01"}
Oct 30 18:37:22.512187 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.512157    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:22.587118 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.587089    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tkz7q_cff1eb5a-0fc0-47ad-adff-12489a13e317/serve-healthcheck-canary/0.log"
Oct 30 18:37:22.924978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:22.924944    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.000363 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.000330    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:23.065358 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.065331    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.114285 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.114242    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:23.155232 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.155203    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:23.155344 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.155308    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:23.155380 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.155348    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:23.155430 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.155414    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:23.155527 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.155503    4938 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 18:37:23.155579 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.155558    4938 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:23.155657 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.155583    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs podName:5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.155560327 +0000 UTC m=+8.915568786 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs") pod "network-metrics-daemon-mlgx9" (UID: "5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc") : secret "metrics-daemon-secret" not found
Oct 30 18:37:23.155657 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.155624    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.155606091 +0000 UTC m=+8.915614550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:23.155737 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.155657    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.155637618 +0000 UTC m=+8.915646074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:23.157497 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.157470    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f35ce557-816c-4b2e-97cc-19937efb59db-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-r6r8p\" (UID: \"f35ce557-816c-4b2e-97cc-19937efb59db\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:23.215822 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.215795    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.315360 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.315325    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"
Oct 30 18:37:23.361632 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.360553    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.484150 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.484052    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6w2g6" event={"ID":"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f","Type":"ContainerStarted","Data":"b04d8329c2533f03e78dc6c3fc9d94a55719e559d053ea4c6458b4f69cf04062"}
Oct 30 18:37:23.484606 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.484148    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6w2g6" event={"ID":"1d135ba2-ec2b-4cc2-a1a9-478ba254e16f","Type":"ContainerStarted","Data":"84917e3118c3e1a2686e585ff30c740f5fb0f1b303a95abe6a38f5adf479f3ff"}
Oct 30 18:37:23.499641 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.499618    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:23.589713 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.589685    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.609424 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.609397    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-r6r8p"]
Oct 30 18:37:23.612451 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:23.612424    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf35ce557_816c_4b2e_97cc_19937efb59db.slice/crio-45e2c482e2ec3b6678befb96d04d6de4bb2e2060f60b8ebfc9140bb6140fd228 WatchSource:0}: Error finding container 45e2c482e2ec3b6678befb96d04d6de4bb2e2060f60b8ebfc9140bb6140fd228: Status 404 returned error can't find the container with id 45e2c482e2ec3b6678befb96d04d6de4bb2e2060f60b8ebfc9140bb6140fd228
Oct 30 18:37:23.776598 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.776522    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:23.976214 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.976182    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:23.976531 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:23.976518    4938 scope.go:122] "RemoveContainer" containerID="a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56"
Oct 30 18:37:23.976699 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:23.976682    4938 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:24.369771 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.369741    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:24.415568 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.415542    4938 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:24.490386 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.490350    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p" event={"ID":"f35ce557-816c-4b2e-97cc-19937efb59db","Type":"ContainerStarted","Data":"45e2c482e2ec3b6678befb96d04d6de4bb2e2060f60b8ebfc9140bb6140fd228"}
Oct 30 18:37:24.490847 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.490682    4938 scope.go:122] "RemoveContainer" containerID="a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56"
Oct 30 18:37:24.490929 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:24.490904    4938 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-rzcch_openshift-console-operator(808b5fa2-8623-4639-870a-be16d38cafeb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb"
Oct 30 18:37:24.637758 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.637686    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:24.975098 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:24.975025    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:25.182355 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:25.182313    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:25.497543 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:25.497504    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-r6r8p" event={"ID":"f35ce557-816c-4b2e-97cc-19937efb59db","Type":"ContainerStarted","Data":"1cb5163f96d63378cbd49b671db328a1068fa5a724ebcbade20b3cbfb04ea531"}
Oct 30 18:37:25.584196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:25.584165    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:25.742189 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:25.742155    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:25.892640 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:25.892606    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:26.027755 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:26.027718    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:26.366328 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:26.366299    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:26.600391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:26.600360    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:26.960537 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:26.960498    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:27.168324 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.168273    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:27.190355 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.190330    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:27.190440 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.190391    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:27.190440 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.190420    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:27.190522 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:27.190508    4938 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:27.190564 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:27.190514    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle podName:666afb05-0236-4db8-8229-0871af2ff16f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:35.190493198 +0000 UTC m=+16.950501644 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle") pod "router-default-c6947b789-btt56" (UID: "666afb05-0236-4db8-8229-0871af2ff16f") : configmap references non-existent config key: service-ca.crt
Oct 30 18:37:27.190564 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:27.190550    4938 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls podName:8feb3890-bb59-4850-a7cf-7c53c9048ad1 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:35.190537408 +0000 UTC m=+16.950545849 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-p9vkt" (UID: "8feb3890-bb59-4850-a7cf-7c53c9048ad1") : secret "insights-runtime-extractor-tls" not found
Oct 30 18:37:27.192681 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.192658    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc-metrics-certs\") pod \"network-metrics-daemon-mlgx9\" (UID: \"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc\") " pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:27.220766 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.220718    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlgx9"
Oct 30 18:37:27.487082 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.487005    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:27.514652 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.514626    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlgx9"]
Oct 30 18:37:27.517612 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:27.517586    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc9b6fb_c39c_4d17_85cd_6ba2f37199fc.slice/crio-aaddf626a8268d65ebef8bc031c6d69665b2453bdddbe7ba1a5ae0468362edf4 WatchSource:0}: Error finding container aaddf626a8268d65ebef8bc031c6d69665b2453bdddbe7ba1a5ae0468362edf4: Status 404 returned error can't find the container with id aaddf626a8268d65ebef8bc031c6d69665b2453bdddbe7ba1a5ae0468362edf4
Oct 30 18:37:27.628691 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.628642    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:27.769039 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:27.768961    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:28.357763 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:28.357737    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:28.509233 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:28.509195    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlgx9" event={"ID":"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc","Type":"ContainerStarted","Data":"db9bb1ee8d30201eabd43d0f41f2d9e7c52df9fd49b5321f3afaab9cbc558704"}
Oct 30 18:37:28.509358 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:28.509242    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlgx9" event={"ID":"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc","Type":"ContainerStarted","Data":"49b1ff33f40180f91ff9b0ec3960428d1413baa073d8226f4dd1547daad3bac0"}
Oct 30 18:37:28.509358 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:28.509298    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlgx9" event={"ID":"5bc9b6fb-c39c-4d17-85cd-6ba2f37199fc","Type":"ContainerStarted","Data":"aaddf626a8268d65ebef8bc031c6d69665b2453bdddbe7ba1a5ae0468362edf4"}
Oct 30 18:37:28.970697 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:28.970664    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:29.172908 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.172871    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:29.285900 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.285825    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:29.286154 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.286139    4938 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:37:29.527003 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.526973    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:29.662343 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.662312    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:29.800465 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:29.800439    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:30.170454 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.170423    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:30.372839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.372809    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:30.388167 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.388142    4938 scope.go:122] "RemoveContainer" containerID="a81f1b80750d43baade6d746c5cd99460ba238e81093f49d0a0c08839f5e5f56"
Oct 30 18:37:30.517862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.517835    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:37:30.518013 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.517918    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" event={"ID":"808b5fa2-8623-4639-870a-be16d38cafeb","Type":"ContainerStarted","Data":"a7ab2f34c920775bb27bcf41ad64d896f7999c867362ea1e6f228bbbb87584db"}
Oct 30 18:37:30.518173 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.518159    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:30.519088 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.519061    4938 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-rzcch container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.18:8443/readyz\": dial tcp 10.133.0.18:8443: connect: connection refused" start-of-body=
Oct 30 18:37:30.519167 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:30.519117    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch" podUID="808b5fa2-8623-4639-870a-be16d38cafeb" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.18:8443/readyz\": dial tcp 10.133.0.18:8443: connect: connection refused"
Oct 30 18:37:31.146052 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.146020    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:31.285346 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.285316    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:31.452204 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.452124    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:31.525935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.525906    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-rzcch"
Oct 30 18:37:31.600645 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.600613    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:31.920007 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:31.919973    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:32.927929 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:32.927894    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:33.069452 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:33.069418    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:34.196969 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:34.196944    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-k5ctz"
Oct 30 18:37:34.499631 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:34.499553    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6w2g6"
Oct 30 18:37:34.655961 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:34.655930    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6bz4h"
Oct 30 18:37:34.783846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:34.783775    4938 patch_prober.go:29] interesting pod/image-registry-5bbf88fc59-82jt6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:37:34.783846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:34.783828    4938 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:35.257616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.257582    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:35.258096 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.257628    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:35.258174 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.258154    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/666afb05-0236-4db8-8229-0871af2ff16f-service-ca-bundle\") pod \"router-default-c6947b789-btt56\" (UID: \"666afb05-0236-4db8-8229-0871af2ff16f\") " pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:35.259926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.259902    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8feb3890-bb59-4850-a7cf-7c53c9048ad1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-p9vkt\" (UID: \"8feb3890-bb59-4850-a7cf-7c53c9048ad1\") " pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:35.313462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.313436    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:35.329079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.329055    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-p9vkt"
Oct 30 18:37:35.640952 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.640930    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-p9vkt"]
Oct 30 18:37:35.643654 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:35.643592    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8feb3890_bb59_4850_a7cf_7c53c9048ad1.slice/crio-414e751684360851b1483e6c19a5a6fd6c716cf0d3facf918a77fefc89705db2 WatchSource:0}: Error finding container 414e751684360851b1483e6c19a5a6fd6c716cf0d3facf918a77fefc89705db2: Status 404 returned error can't find the container with id 414e751684360851b1483e6c19a5a6fd6c716cf0d3facf918a77fefc89705db2
Oct 30 18:37:35.646157 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:35.646133    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod666afb05_0236_4db8_8229_0871af2ff16f.slice/crio-8ae3d68690012fb288118401a05d13dd097df145beb652f68bb9bf445a386793 WatchSource:0}: Error finding container 8ae3d68690012fb288118401a05d13dd097df145beb652f68bb9bf445a386793: Status 404 returned error can't find the container with id 8ae3d68690012fb288118401a05d13dd097df145beb652f68bb9bf445a386793
Oct 30 18:37:35.654029 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.654009    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c6947b789-btt56"]
Oct 30 18:37:35.764914 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:35.764880    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:36.541831 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.541752    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9vkt" event={"ID":"8feb3890-bb59-4850-a7cf-7c53c9048ad1","Type":"ContainerStarted","Data":"0cac978081f64b15ca61cc1b814c3c0e21d9041188e611a3d4152a1f33966659"}
Oct 30 18:37:36.541831 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.541807    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9vkt" event={"ID":"8feb3890-bb59-4850-a7cf-7c53c9048ad1","Type":"ContainerStarted","Data":"3088c13a98c8f1cd8ae6319cb37e41c46604a34e102f78bbd64b60eea99e36cc"}
Oct 30 18:37:36.542311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.541831    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9vkt" event={"ID":"8feb3890-bb59-4850-a7cf-7c53c9048ad1","Type":"ContainerStarted","Data":"414e751684360851b1483e6c19a5a6fd6c716cf0d3facf918a77fefc89705db2"}
Oct 30 18:37:36.543197 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.543175    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c6947b789-btt56" event={"ID":"666afb05-0236-4db8-8229-0871af2ff16f","Type":"ContainerStarted","Data":"ca0625330157a74091d2c0ddfc96b1440ef74f94285ff363c7a1d25418bf819a"}
Oct 30 18:37:36.543261 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.543213    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c6947b789-btt56" event={"ID":"666afb05-0236-4db8-8229-0871af2ff16f","Type":"ContainerStarted","Data":"8ae3d68690012fb288118401a05d13dd097df145beb652f68bb9bf445a386793"}
Oct 30 18:37:36.605312 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.605283    4938 patch_prober.go:29] interesting pod/image-registry-5bbf88fc59-82jt6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:37:36.605456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:36.605338    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:37.314309 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:37.314272    4938 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:37.317164 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:37.317137    4938 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:37.547052 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:37.547024    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:37.548126 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:37.548108    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c6947b789-btt56"
Oct 30 18:37:38.552851 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:38.552816    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-p9vkt" event={"ID":"8feb3890-bb59-4850-a7cf-7c53c9048ad1","Type":"ContainerStarted","Data":"3257810273edd421dbeb89fe90c4787d5a75e5507f919d2b78c258cc03b4bb31"}
Oct 30 18:37:43.006658 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:43.006615    4938 patch_prober.go:29] interesting pod/image-registry-8457959978-p9454 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:37:43.007298 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:43.006670    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-8457959978-p9454" podUID="82214171-38b1-45d5-8c9b-794a142335fc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:44.424242 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:44.424202    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:44.579837 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:44.579800    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:44.715434 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:44.715361    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:44.780924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:44.780896    4938 patch_prober.go:29] interesting pod/image-registry-5bbf88fc59-82jt6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:37:44.781067 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:44.780948    4938 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:45.289641 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.289606    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:45.428552 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.428490    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:45.482205 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.482172    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8457959978-p9454" podUID="82214171-38b1-45d5-8c9b-794a142335fc" containerName="registry" containerID="cri-o://f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d" gracePeriod=30
Oct 30 18:37:45.741895 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.741870    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:45.819914 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.819879    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.819920    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.819951    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.819976    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.820011    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.820050    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820382 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.820129    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.820382 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.820158    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") pod \"82214171-38b1-45d5-8c9b-794a142335fc\" (UID: \"82214171-38b1-45d5-8c9b-794a142335fc\") "
Oct 30 18:37:45.821011 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.820694    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:45.821309 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.821278    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:37:45.822586 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.822548    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:45.822770 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.822751    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:45.822907 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.822887    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:45.823646 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.823624    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:37:45.824804 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.824778    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "kube-api-access-5zjrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:37:45.828801 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.828777    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted" pod "82214171-38b1-45d5-8c9b-794a142335fc" (UID: "82214171-38b1-45d5-8c9b-794a142335fc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921220    4938 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-registry-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921269    4938 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/82214171-38b1-45d5-8c9b-794a142335fc-ca-trust-extracted\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921282    4938 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-installation-pull-secrets\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921292    4938 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-bound-sa-token\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921301    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zjrb\" (UniqueName: \"kubernetes.io/projected/82214171-38b1-45d5-8c9b-794a142335fc-kube-api-access-5zjrb\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921310    4938 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-registry-certificates\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921320    4938 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/82214171-38b1-45d5-8c9b-794a142335fc-image-registry-private-configuration\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:45.921577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:45.921330    4938 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/82214171-38b1-45d5-8c9b-794a142335fc-trusted-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:37:46.584952 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.584919    4938 generic.go:356] "Generic (PLEG): container finished" podID="82214171-38b1-45d5-8c9b-794a142335fc" containerID="f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d" exitCode=0
Oct 30 18:37:46.585379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.584981    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8457959978-p9454"
Oct 30 18:37:46.585379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.585003    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8457959978-p9454" event={"ID":"82214171-38b1-45d5-8c9b-794a142335fc","Type":"ContainerDied","Data":"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"}
Oct 30 18:37:46.585379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.585053    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8457959978-p9454" event={"ID":"82214171-38b1-45d5-8c9b-794a142335fc","Type":"ContainerDied","Data":"4a3ec049f55646adb44724850db3a64e4a288d7df7482cf70a3eab7b05333d50"}
Oct 30 18:37:46.585379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.585076    4938 scope.go:122] "RemoveContainer" containerID="f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"
Oct 30 18:37:46.592815 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.592797    4938 scope.go:122] "RemoveContainer" containerID="f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"
Oct 30 18:37:46.593095 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:37:46.593074    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d\": container with ID starting with f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d not found: ID does not exist" containerID="f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"
Oct 30 18:37:46.593139 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.593104    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d"} err="failed to get container status \"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d\": rpc error: code = NotFound desc = could not find container \"f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d\": container with ID starting with f9a066609243df7d3add83b9a43a4c37abd74d464ef97bc8e57dedc0db39b83d not found: ID does not exist"
Oct 30 18:37:46.605182 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.605159    4938 patch_prober.go:29] interesting pod/image-registry-5bbf88fc59-82jt6 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 30 18:37:46.605304 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.605205    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:37:46.650952 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.650886    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:46.663792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:46.663769    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8457959978-p9454"]
Oct 30 18:37:48.391888 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:48.391852    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82214171-38b1-45d5-8c9b-794a142335fc" path="/var/lib/kubelet/pods/82214171-38b1-45d5-8c9b-794a142335fc/volumes"
Oct 30 18:37:48.845325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:48.845295    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.064447 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.064412    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.105328 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.105238    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tfjt5"]
Oct 30 18:37:49.105582 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.105568    4938 cpu_manager.go:414] "containerMap: removing container" podUID="82214171-38b1-45d5-8c9b-794a142335fc" containerName="registry"
Oct 30 18:37:49.105622 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.105584    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82214171-38b1-45d5-8c9b-794a142335fc" containerName="registry"
Oct 30 18:37:49.105658 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.105651    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="82214171-38b1-45d5-8c9b-794a142335fc" containerName="registry"
Oct 30 18:37:49.137166 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.137138    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144286 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144237    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-root\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144398 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144348    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-wtmp\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144398 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144374    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144400    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvmf\" (UniqueName: \"kubernetes.io/projected/cd97b741-9720-4db9-811d-0fc9ef8fef22-kube-api-access-bgvmf\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144441    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-tls\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144459    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-metrics-client-ca\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144584 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144480    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-textfile\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144584 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144499    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.144584 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.144516    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-sys\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.165566 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.165537    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:37:49.165697 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.165588    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:37:49.171379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.171354    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zvl8z\""
Oct 30 18:37:49.171601 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.171579    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:37:49.175614 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.175598    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:37:49.208457 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.208427    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.245519 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245486    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-root\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245527    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-wtmp\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245552    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245580    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvmf\" (UniqueName: \"kubernetes.io/projected/cd97b741-9720-4db9-811d-0fc9ef8fef22-kube-api-access-bgvmf\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245603    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-root\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245616    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-tls\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245637    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-metrics-client-ca\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245691    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-wtmp\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245748    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-textfile\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245792    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245827    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-sys\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.245978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.245913    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd97b741-9720-4db9-811d-0fc9ef8fef22-sys\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.246159 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.246121    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-textfile\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.246482 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.246460    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-metrics-client-ca\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.246542 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.246461    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-accelerators-collector-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.248000 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.247982    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-tls\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.248087 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.248069    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cd97b741-9720-4db9-811d-0fc9ef8fef22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.267755 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.267729    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvmf\" (UniqueName: \"kubernetes.io/projected/cd97b741-9720-4db9-811d-0fc9ef8fef22-kube-api-access-bgvmf\") pod \"node-exporter-tfjt5\" (UID: \"cd97b741-9720-4db9-811d-0fc9ef8fef22\") " pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.367636 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.367560    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.487778 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.487748    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tfjt5"
Oct 30 18:37:49.509496 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.509466    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:49.529096 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:37:49.529067    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd97b741_9720_4db9_811d_0fc9ef8fef22.slice/crio-9b93b34ac9860659989e1c0b2fa607c61aad584130a60b2eae21c2ee8466b731 WatchSource:0}: Error finding container 9b93b34ac9860659989e1c0b2fa607c61aad584130a60b2eae21c2ee8466b731: Status 404 returned error can't find the container with id 9b93b34ac9860659989e1c0b2fa607c61aad584130a60b2eae21c2ee8466b731
Oct 30 18:37:49.598215 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.598188    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfjt5" event={"ID":"cd97b741-9720-4db9-811d-0fc9ef8fef22","Type":"ContainerStarted","Data":"9b93b34ac9860659989e1c0b2fa607c61aad584130a60b2eae21c2ee8466b731"}
Oct 30 18:37:49.657264 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:49.657181    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:50.371066 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:50.371036    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:50.582844 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:50.582806    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:50.846543 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:50.846498    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:51.004511 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:51.004479    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:51.608512 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:51.608473    4938 generic.go:356] "Generic (PLEG): container finished" podID="cd97b741-9720-4db9-811d-0fc9ef8fef22" containerID="9a0704ea7405f714d27167fc2e35a9831bffc903c71f66621f03a94f8f638219" exitCode=0
Oct 30 18:37:51.608982 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:51.608551    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfjt5" event={"ID":"cd97b741-9720-4db9-811d-0fc9ef8fef22","Type":"ContainerDied","Data":"9a0704ea7405f714d27167fc2e35a9831bffc903c71f66621f03a94f8f638219"}
Oct 30 18:37:51.934422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:51.934351    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:52.075026 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.074997    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:52.408357 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.408322    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:52.566473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.566436    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:52.614444 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.614409    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfjt5" event={"ID":"cd97b741-9720-4db9-811d-0fc9ef8fef22","Type":"ContainerStarted","Data":"4875c65284ccb54646a12ce8cda1044dc76b62dc2c9dbc50d8d326b2f550ed0c"}
Oct 30 18:37:52.614831 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.614468    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tfjt5" event={"ID":"cd97b741-9720-4db9-811d-0fc9ef8fef22","Type":"ContainerStarted","Data":"a0f137506f3b5afb60309cdae5df00c9c0e919660ebfd3caeaaddd285c87d075"}
Oct 30 18:37:52.687386 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.686060    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tfjt5" podStartSLOduration=2.091745148 podStartE2EDuration="3.686043379s" totalImagesPullingTime="1.513625811s" totalInitContainerRuntime="80.67242ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:52.672083955 +0000 UTC m=+34.432092419" watchObservedRunningTime="2026-10-30 18:37:52.686043379 +0000 UTC m=+34.446051850"
Oct 30 18:37:52.718886 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:52.718860    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:53.117076 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:53.117044    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kps9t"
Oct 30 18:37:53.956210 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:53.956181    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:54.099287 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:54.099258    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:55.961827 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:55.961786    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:56.130110 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:56.130078    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:56.536687 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:56.536655    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:56.605999 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:56.605929    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:37:56.686120 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:56.686093    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:57.011475 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:57.011438    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:57.151977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:57.151938    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:59.129994 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:59.129962    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:59.262634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:59.262603    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:59.854070 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:59.854035    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:37:59.951842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:37:59.951810    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:38:00.088130 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:00.088089    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:00.230623 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:00.230547    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:00.373946 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:00.373913    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:00.715928 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:00.715896    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:01.740283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:01.740232    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:01.877449 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:01.877396    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:02.549321 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:38:02.549287    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:38:04.331913 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:04.331862    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:04.467733 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:04.467697    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:06.934688 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:06.934655    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:07.072136 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:07.072104    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:07.233123 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:07.233051    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:07.371707 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:07.371670    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:08.388789 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:08.388754    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:08.554937 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:08.554906    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:14.522117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:14.522084    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:14.665816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:14.665778    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:15.941683 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:15.941650    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:16.093651 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:16.093607    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:21.649142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:21.649108    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:21.999162 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:21.999086    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:22.167754 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:22.167718    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:22.730699 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:22.730670    4938 generic.go:356] "Generic (PLEG): container finished" podID="e27601fc-9881-482b-9115-a169779efa5b" containerID="3aedc4fc90d43ce22b56809f65eae27e434f2343c5520626794a5353f8ea9f14" exitCode=0
Oct 30 18:38:22.731023 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:22.730739    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8" event={"ID":"e27601fc-9881-482b-9115-a169779efa5b","Type":"ContainerDied","Data":"3aedc4fc90d43ce22b56809f65eae27e434f2343c5520626794a5353f8ea9f14"}
Oct 30 18:38:22.731066 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:22.731035    4938 scope.go:122] "RemoveContainer" containerID="3aedc4fc90d43ce22b56809f65eae27e434f2343c5520626794a5353f8ea9f14"
Oct 30 18:38:23.737083 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:23.737045    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-z7rk8" event={"ID":"e27601fc-9881-482b-9115-a169779efa5b","Type":"ContainerStarted","Data":"dcac8e9be1b9e1562039b8fb8359231655d4a40176b665aae386e737eca8aea5"}
Oct 30 18:38:24.977464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:24.977399    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry" containerID="cri-o://4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb" gracePeriod=30
Oct 30 18:38:25.224420 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.224395    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:38:25.337148 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337116    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337152    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337195    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337229    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337296    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337325 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337317    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337607 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337343    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337607 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337367    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") pod \"50f0c5e2-b263-4047-b980-20b8ebb07a79\" (UID: \"50f0c5e2-b263-4047-b980-20b8ebb07a79\") "
Oct 30 18:38:25.337878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.337843    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:25.338395 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.338291    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:38:25.339725 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.339693    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "kube-api-access-ft5kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:25.339899 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.339859    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:25.339988 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.339967    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:25.340047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.339972    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:38:25.340084 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.340053    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:38:25.344391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.344369    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted" pod "50f0c5e2-b263-4047-b980-20b8ebb07a79" (UID: "50f0c5e2-b263-4047-b980-20b8ebb07a79"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:38:25.437829 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437805    4938 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-certificates\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437829 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437825    4938 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50f0c5e2-b263-4047-b980-20b8ebb07a79-ca-trust-extracted\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437835    4938 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50f0c5e2-b263-4047-b980-20b8ebb07a79-trusted-ca\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437845    4938 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-installation-pull-secrets\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437853    4938 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-bound-sa-token\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437862    4938 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50f0c5e2-b263-4047-b980-20b8ebb07a79-image-registry-private-configuration\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437872    4938 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-registry-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.437960 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.437880    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft5kv\" (UniqueName: \"kubernetes.io/projected/50f0c5e2-b263-4047-b980-20b8ebb07a79-kube-api-access-ft5kv\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:38:25.745877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.745796    4938 generic.go:356] "Generic (PLEG): container finished" podID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerID="4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb" exitCode=0
Oct 30 18:38:25.745877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.745844    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6"
Oct 30 18:38:25.746063 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.745878    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" event={"ID":"50f0c5e2-b263-4047-b980-20b8ebb07a79","Type":"ContainerDied","Data":"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"}
Oct 30 18:38:25.746063 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.745932    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bbf88fc59-82jt6" event={"ID":"50f0c5e2-b263-4047-b980-20b8ebb07a79","Type":"ContainerDied","Data":"9dc9e68ad5a93261b430e11935855803580c8c8e14bbc22fc435b4c4952bc3d4"}
Oct 30 18:38:25.746063 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.745954    4938 scope.go:122] "RemoveContainer" containerID="4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"
Oct 30 18:38:25.757813 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.757796    4938 scope.go:122] "RemoveContainer" containerID="4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"
Oct 30 18:38:25.758211 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:38:25.758195    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb\": container with ID starting with 4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb not found: ID does not exist" containerID="4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"
Oct 30 18:38:25.758281 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.758220    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb"} err="failed to get container status \"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb\": rpc error: code = NotFound desc = could not find container \"4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb\": container with ID starting with 4153cacece16d74e5331da184ad1bfd4d3754bb760aeb51794ffca85339aa6fb not found: ID does not exist"
Oct 30 18:38:25.817764 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.817742    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:38:25.831176 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:25.831150    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5bbf88fc59-82jt6"]
Oct 30 18:38:26.391985 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:26.391948    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" path="/var/lib/kubelet/pods/50f0c5e2-b263-4047-b980-20b8ebb07a79/volumes"
Oct 30 18:38:29.518076 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:29.518036    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:29.657286 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:29.657236    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:29.820572 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:29.820538    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:29.959704 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:29.959671    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:30.884424 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:30.884393    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:32.548529 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:38:32.548490    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:38:38.790711 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:38.790678    4938 generic.go:356] "Generic (PLEG): container finished" podID="77ce2b35-2046-4b1a-bb36-7f49fe33f506" containerID="2d1b39522cd12e17761ea29bbb9f3e92c9db505c8bf467f6a4d804fbd48e0faa" exitCode=0
Oct 30 18:38:38.791087 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:38.790759    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9" event={"ID":"77ce2b35-2046-4b1a-bb36-7f49fe33f506","Type":"ContainerDied","Data":"2d1b39522cd12e17761ea29bbb9f3e92c9db505c8bf467f6a4d804fbd48e0faa"}
Oct 30 18:38:38.791087 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:38.791066    4938 scope.go:122] "RemoveContainer" containerID="2d1b39522cd12e17761ea29bbb9f3e92c9db505c8bf467f6a4d804fbd48e0faa"
Oct 30 18:38:39.796563 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:39.796530    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-psxz9" event={"ID":"77ce2b35-2046-4b1a-bb36-7f49fe33f506","Type":"ContainerStarted","Data":"25fd95e63f6d7a7865fa1c1666335bba5e421c1b8bd0110921c40399b08bfe83"}
Oct 30 18:38:41.863747 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:41.863704    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:42.017427 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:42.017389    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:49.941628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:49.941591    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:38:53.846409 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:53.846375    4938 generic.go:356] "Generic (PLEG): container finished" podID="bf0111a4-adeb-451d-a0c2-17469dce5b90" containerID="e68823097d682ec7b08f72b174c2ceaf15248c1b02204d57dca4557de2b531bf" exitCode=0
Oct 30 18:38:53.846816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:53.846451    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-mvhgb" event={"ID":"bf0111a4-adeb-451d-a0c2-17469dce5b90","Type":"ContainerDied","Data":"e68823097d682ec7b08f72b174c2ceaf15248c1b02204d57dca4557de2b531bf"}
Oct 30 18:38:53.846816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:53.846788    4938 scope.go:122] "RemoveContainer" containerID="e68823097d682ec7b08f72b174c2ceaf15248c1b02204d57dca4557de2b531bf"
Oct 30 18:38:54.852303 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:38:54.852269    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-mvhgb" event={"ID":"bf0111a4-adeb-451d-a0c2-17469dce5b90","Type":"ContainerStarted","Data":"5398832e8ed10229cd6f70e353163cd748db3d0fc59fe6f9a1fe43c359b32f22"}
Oct 30 18:39:02.548213 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:39:02.548169    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:39:08.473613 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:08.473569    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:25.934184 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:25.934151    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:26.076356 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:26.076322    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:31.358545 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:31.358463    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:31.506441 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:31.506411    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:32.549776 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:39:32.549745    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:39:33.012305 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:33.012272    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:33.157470 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:33.157437    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:38.594772 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:38.594741    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:38.735899 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:38.735868    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:44.524549 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:44.524515    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:39:44.664957 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:39:44.664923    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:02.549021 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:40:02.548973    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:40:08.629717 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:40:08.629682    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:21.963852 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:40:21.963817    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:22.098790 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:40:22.098759    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:32.549624 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:40:32.549585    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:40:54.628203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:40:54.628168    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:40:54.767740 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:40:54.767699    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:02.549239 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:41:02.549145    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:41:08.747883 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:41:08.747848    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:28.849947 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:41:28.849878    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:28.984613 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:41:28.984565    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:41:32.551280 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:41:32.551221    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:42:02.549820 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:42:02.549778    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:42:08.870401 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:08.870361    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:18.414967 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:18.414934    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:42:18.416771 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:18.416747    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:42:18.419044 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:18.419022    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:42:18.420668 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:18.420648    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:42:18.974159 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:18.974121    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:32.549513 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:42:32.549418    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:42:33.142380 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:33.142341    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:43.226608 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:43.226567    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:44.642233 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:44.642193    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:44.779709 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:44.779668    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:42:44.932230 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:42:44.932137    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:02.548843 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:43:02.548799    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:43:07.728108 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:07.728072    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:07.877925 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:07.877886    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:08.996158 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:08.996121    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:13.924526 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:13.924491    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:14.554175 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:14.554140    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:14.707025 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:14.706977    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:15.414566 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:15.414525    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:15.671428 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:15.671347    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:15.809071 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:15.809036    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:15.954466 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:15.954383    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:16.449668 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:16.449626    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:16.794302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:16.794185    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.058574 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:17.058470    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:17.210121 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:17.210089    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:27.745450 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:27.745411    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:27.884300 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:27.884262    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:43:32.549985 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:43:32.549944    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:43:43.229816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:43:43.229779    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:02.549390 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:44:02.549305    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:44:09.127924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:44:09.127884    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:32.549554 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:44:32.549497    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:44:49.642687 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:44:49.642640    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:44:49.807097 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:44:49.807058    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:02.549939 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:45:02.549898    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:45:09.282040 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:09.282004    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:21.611212 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.611171    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:21.713816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.713783    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"]
Oct 30 18:45:21.714090 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.714077    4938 cpu_manager.go:414] "containerMap: removing container" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry"
Oct 30 18:45:21.714132 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.714092    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry"
Oct 30 18:45:21.714181 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.714171    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="50f0c5e2-b263-4047-b980-20b8ebb07a79" containerName="registry"
Oct 30 18:45:21.717104 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.717080    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.753297 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.753258    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"]
Oct 30 18:45:21.775835 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.775791    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:21.816275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.816228    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Oct 30 18:45:21.816275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.816228    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-ngjtq\""
Oct 30 18:45:21.816275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.816272    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 18:45:21.816552 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.816387    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 18:45:21.828491 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.828461    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ff7b5f-776b-49a6-86d0-c91e34a477aa-cert\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.828584 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.828516    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l96cl\" (UniqueName: \"kubernetes.io/projected/68ff7b5f-776b-49a6-86d0-c91e34a477aa-kube-api-access-l96cl\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.929875 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.929775    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l96cl\" (UniqueName: \"kubernetes.io/projected/68ff7b5f-776b-49a6-86d0-c91e34a477aa-kube-api-access-l96cl\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.930051 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.929937    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ff7b5f-776b-49a6-86d0-c91e34a477aa-cert\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.932387 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.932367    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ff7b5f-776b-49a6-86d0-c91e34a477aa-cert\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.951547 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.951512    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l96cl\" (UniqueName: \"kubernetes.io/projected/68ff7b5f-776b-49a6-86d0-c91e34a477aa-kube-api-access-l96cl\") pod \"llmisvc-controller-manager-67dc8bf7db-2f6r6\" (UID: \"68ff7b5f-776b-49a6-86d0-c91e34a477aa\") " pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:21.983775 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:21.983741    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:22.029859 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:22.029825    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:22.129767 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:22.129730    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:22.327531 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:22.327493    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"]
Oct 30 18:45:22.331345 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:45:22.331306    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ff7b5f_776b_49a6_86d0_c91e34a477aa.slice/crio-1118755241d344213b4f55ea7b26aa4fc6aeed17d2ae355467c00cb371e3440e WatchSource:0}: Error finding container 1118755241d344213b4f55ea7b26aa4fc6aeed17d2ae355467c00cb371e3440e: Status 404 returned error can't find the container with id 1118755241d344213b4f55ea7b26aa4fc6aeed17d2ae355467c00cb371e3440e
Oct 30 18:45:22.333033 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:22.333013    4938 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:45:22.621153 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:22.621063    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:23.140422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:23.140351    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:23.240398 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:23.240362    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6" event={"ID":"68ff7b5f-776b-49a6-86d0-c91e34a477aa","Type":"ContainerStarted","Data":"1118755241d344213b4f55ea7b26aa4fc6aeed17d2ae355467c00cb371e3440e"}
Oct 30 18:45:23.290923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:23.290865    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:23.440183 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:23.440100    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:45:25.256880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:25.256842    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6" event={"ID":"68ff7b5f-776b-49a6-86d0-c91e34a477aa","Type":"ContainerStarted","Data":"5abadf7d2ca19f96804ba15952343d787cce5fd57b60114a314ab1401f83b9ff"}
Oct 30 18:45:25.257307 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:25.256964    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:32.551515 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:45:32.551428    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:45:56.264064 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:56.264028    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6"
Oct 30 18:45:56.316547 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:45:56.316495    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-67dc8bf7db-2f6r6" podStartSLOduration=32.896493105 podStartE2EDuration="35.316481952s" totalImagesPullingTime="2.419988847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:25.326111618 +0000 UTC m=+487.086120080" watchObservedRunningTime="2026-10-30 18:45:56.316481952 +0000 UTC m=+518.076490415"
Oct 30 18:46:02.550287 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:46:02.550234    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:46:09.388620 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:09.388589    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.549557 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:46:32.549501    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:46:44.833574 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:44.833540    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:44.976436 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:44.976399    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:45.126865 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:45.126772    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:48.720322 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:48.720287    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:49.277678 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:49.277643    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:49.418880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:46:49.418849    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:02.549633 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:47:02.549594    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:47:09.487797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:47:09.487760    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:18.440495 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:47:18.440462    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:47:18.442717 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:47:18.442693    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:47:18.444694 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:47:18.444666    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:47:18.446872 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:47:18.446854    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:47:32.549952 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:47:32.549912    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:48:02.549813 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:48:02.549768    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:48:03.825329 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:03.825290    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:09.602410 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:09.602373    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:20.015602 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:20.015567    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:20.154578 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:20.154544    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:32.548826 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:48:32.548740    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:48:55.854357 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:55.854319    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:56.004733 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:56.004699    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.390234 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.390203    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.555210 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.555173    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.708845 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.708768    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.832160 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.832119    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:48:57.835661 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.835637    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:57.865509 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.865475    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:57.867594 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.867566    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:48:57.897531 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.897493    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"seaweedfs-s3-creds\""
Oct 30 18:48:57.902026 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.901998    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:48:57.902142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.902004    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 18:48:57.902142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.902062    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 18:48:57.902142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.902063    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Oct 30 18:48:57.902142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.902077    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-canary-rollback-predictor-serving-cert\""
Oct 30 18:48:57.902142 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.902089    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"default-dockercfg-gh4qc\""
Oct 30 18:48:57.949494 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.949453    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:57.949660 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.949509    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:57.949660 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.949617    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:57.949747 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.949653    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:57.949747 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:57.949724    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpvbv\" (UniqueName: \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051057    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpvbv\" (UniqueName: \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051119    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051200    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051231    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051305    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.051801 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.051775    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.052061 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.052036    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.052111 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.052036    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.053932 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.053908    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.072323 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.072280    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpvbv\" (UniqueName: \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\") pod \"isvc-canary-rollback-predictor-95869db44-vzgpg\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.091427 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.091392    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:58.152423 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.152381    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:48:58.237542 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.237512    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:48:58.241334 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.241318    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.261985 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.261958    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 18:48:58.262152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.262055    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-raw-sklearn-batcher-cae97-kube-rbac-proxy-sar-config\""
Oct 30 18:48:58.262152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.262061    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 18:48:58.262152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.262119    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:48:58.263330 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.263313    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 18:48:58.263391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.263351    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-raw-sklearn-batcher-cae97-predictor-serving-cert\""
Oct 30 18:48:58.263391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.263385    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 18:48:58.269641 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.269621    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:48:58.353671 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.353628    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.353856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.353694    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vdqz\" (UniqueName: \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.353856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.353769    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.353856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.353821    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.353984 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.353869    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.356492 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.356459    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:58.455119 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455075    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455618 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455132    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455618 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455157    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vdqz\" (UniqueName: \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455618 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455289    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455618 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455441    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455815 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455797    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455921 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455904    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.455969 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.455909    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.457768 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.457746    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.467914 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.467886    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:48:58.471170 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:48:58.471144    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be248a9_64d7_4f5c_b128_838653806aff.slice/crio-ec1b0e61c320cc6ef611a89a606f47411801569a69edbf38aeb5b447d42e7e78 WatchSource:0}: Error finding container ec1b0e61c320cc6ef611a89a606f47411801569a69edbf38aeb5b447d42e7e78: Status 404 returned error can't find the container with id ec1b0e61c320cc6ef611a89a606f47411801569a69edbf38aeb5b447d42e7e78
Oct 30 18:48:58.476661 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.476641    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vdqz\" (UniqueName: \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\") pod \"isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") " pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.561033 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.560987    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:48:58.876784 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:58.876760    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:48:58.878868 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:48:58.878840    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce9b27a_af0d_4408_b985_5d2af7061d66.slice/crio-15e3c66c1085c11817c1aa6e509d151faf84e934ea2d0298d55138ca008af87e WatchSource:0}: Error finding container 15e3c66c1085c11817c1aa6e509d151faf84e934ea2d0298d55138ca008af87e: Status 404 returned error can't find the container with id 15e3c66c1085c11817c1aa6e509d151faf84e934ea2d0298d55138ca008af87e
Oct 30 18:48:59.035840 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:59.035796    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerStarted","Data":"ec1b0e61c320cc6ef611a89a606f47411801569a69edbf38aeb5b447d42e7e78"}
Oct 30 18:48:59.037306 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:48:59.037278    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerStarted","Data":"15e3c66c1085c11817c1aa6e509d151faf84e934ea2d0298d55138ca008af87e"}
Oct 30 18:49:02.549845 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:49:02.549798    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:49:03.058619 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:03.058568    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerStarted","Data":"df088432d2f9fe6ed3f230d8dfc436217bfb0cbe4ea9393d30bc59e509a95657"}
Oct 30 18:49:03.060366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:03.060329    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerStarted","Data":"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"}
Oct 30 18:49:07.078588 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:07.078554    4938 generic.go:356] "Generic (PLEG): container finished" podID="8be248a9-64d7-4f5c-b128-838653806aff" containerID="df088432d2f9fe6ed3f230d8dfc436217bfb0cbe4ea9393d30bc59e509a95657" exitCode=0
Oct 30 18:49:07.079069 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:07.078628    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerDied","Data":"df088432d2f9fe6ed3f230d8dfc436217bfb0cbe4ea9393d30bc59e509a95657"}
Oct 30 18:49:07.080287 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:07.080242    4938 generic.go:356] "Generic (PLEG): container finished" podID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerID="26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51" exitCode=0
Oct 30 18:49:07.080287 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:07.080280    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerDied","Data":"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"}
Oct 30 18:49:09.743829 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:09.743748    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:27.198433 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:27.198288    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerStarted","Data":"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"}
Oct 30 18:49:27.201314 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:27.201220    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerStarted","Data":"cc3f5cb5858bac1e249c44c90896be464704abb3505a592ae04733c28eb0c207"}
Oct 30 18:49:30.217473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:30.217430    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerStarted","Data":"74c4a7abef9df050b1764b56913287a0ec570a4e8bdfb2cc668dd828161659f1"}
Oct 30 18:49:30.217993 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:30.217541    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:49:30.219761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:30.219727    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerStarted","Data":"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"}
Oct 30 18:49:30.277400 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:30.277349    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podStartSLOduration=2.803957009 podStartE2EDuration="33.277334731s" totalImagesPullingTime="26.443672707s" totalInitContainerRuntime="4.029705015s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:30.275945266 +0000 UTC m=+732.035953730" watchObservedRunningTime="2026-10-30 18:49:30.277334731 +0000 UTC m=+732.037343190"
Oct 30 18:49:31.225334 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:31.225297    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:49:31.226616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:31.226583    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:49:32.229003 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:32.228953    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:49:32.549140 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:49:32.549051    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:49:33.235524 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:33.235486    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerStarted","Data":"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"}
Oct 30 18:49:33.236010 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:33.235712    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:49:33.236010 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:33.235856    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:49:33.237107 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:33.237081    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:33.310211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:33.310154    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podStartSLOduration=2.706665458 podStartE2EDuration="35.31013555s" totalImagesPullingTime="28.879839912s" totalInitContainerRuntime="3.72363018s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:58 +0000 UTC" imagePullSessionsCount=4 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:33.296767582 +0000 UTC m=+735.056776045" watchObservedRunningTime="2026-10-30 18:49:33.31013555 +0000 UTC m=+735.070144016"
Oct 30 18:49:34.239464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:34.239430    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:49:34.239983 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:34.239934    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:34.240614 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:34.240585    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:34.243447 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:34.243428    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:49:35.243976 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:35.243936    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:35.244509 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:35.244288    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:36.248125 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:36.248082    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:36.248569 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:36.248344    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:37.233397 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:37.233367    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:49:37.233991 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:37.233967    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:49:46.248243 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:46.248187    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:46.248833 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:46.248637    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:47.234658 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:47.234618    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:49:56.248358 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:56.248309    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:49:56.248842 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:56.248781    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:49:57.234807 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:57.234765    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:49:57.297904 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:49:57.297871    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:02.549435 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:50:02.549344    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:50:06.248779 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:06.248727    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:50:06.249268 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:06.249222    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:07.234022 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:07.233981    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:50:09.833770 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:09.833736    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:16.248230 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:16.248181    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:50:16.248727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:16.248702    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:16.949426 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:16.949387    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:17.234920 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:17.234831    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:50:26.248143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:26.248092    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:50:26.248603 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:26.248543    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:27.234795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:27.234756    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:50:32.548447 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:50:32.548402    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:50:36.248629 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:36.248582    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:50:36.249164 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:36.249138    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:37.233990 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:37.233945    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:50:41.388392 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:41.388349    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:50:41.388866 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:41.388677    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:50:43.175155 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.175117    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:50:43.175662 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.175365    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" containerID="cri-o://cc3f5cb5858bac1e249c44c90896be464704abb3505a592ae04733c28eb0c207" gracePeriod=30
Oct 30 18:50:43.175662 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.175426    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" containerID="cri-o://74c4a7abef9df050b1764b56913287a0ec570a4e8bdfb2cc668dd828161659f1" gracePeriod=30
Oct 30 18:50:43.175662 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.175596    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Oct 30 18:50:43.525632 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.525542    4938 generic.go:356] "Generic (PLEG): container finished" podID="8be248a9-64d7-4f5c-b128-838653806aff" containerID="74c4a7abef9df050b1764b56913287a0ec570a4e8bdfb2cc668dd828161659f1" exitCode=0
Oct 30 18:50:43.525632 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:43.525613    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerDied","Data":"74c4a7abef9df050b1764b56913287a0ec570a4e8bdfb2cc668dd828161659f1"}
Oct 30 18:50:47.229730 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.229684    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:50:47.776010 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.775971    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:47.842388 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.842351    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:50:47.846013 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.845996    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.868870 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.868840    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.869027 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.868894    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.869075 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.869016    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.869113 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.869082    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.869195 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.869174    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nd72\" (UniqueName: \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.875855 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.875826    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:50:47.970057 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.969999    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970057 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970069    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970099    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970123    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nd72\" (UniqueName: \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970151    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970551 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970516    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970785 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970762    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.970859 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.970825    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.972630 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.972612    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:47.991034 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:47.991006    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nd72\" (UniqueName: \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\") pod \"isvc-canary-rollback-predictor-95869db44-dpk2t\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:48.163589 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:48.163548    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:48.474342 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:48.474310    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:50:48.476990 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:50:48.476959    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod069cd92b_8442_4f41_bb9b_9dae8c94fdec.slice/crio-ecaa3de286a7a57915dfc10ddc6e0eace2ad607e4ef6f4799921affa0f899187 WatchSource:0}: Error finding container ecaa3de286a7a57915dfc10ddc6e0eace2ad607e4ef6f4799921affa0f899187: Status 404 returned error can't find the container with id ecaa3de286a7a57915dfc10ddc6e0eace2ad607e4ef6f4799921affa0f899187
Oct 30 18:50:48.546609 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:48.546567    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerStarted","Data":"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"}
Oct 30 18:50:48.546776 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:48.546651    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerStarted","Data":"ecaa3de286a7a57915dfc10ddc6e0eace2ad607e4ef6f4799921affa0f899187"}
Oct 30 18:50:51.389456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:51.389418    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:50:51.389902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:51.389524    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:50:52.229718 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:52.229674    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:50:52.565476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:52.565391    4938 generic.go:356] "Generic (PLEG): container finished" podID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerID="f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd" exitCode=0
Oct 30 18:50:52.565476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:52.565446    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerDied","Data":"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"}
Oct 30 18:50:53.176455 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.176422    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:50:53.571270 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.571228    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerStarted","Data":"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"}
Oct 30 18:50:53.571688 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.571302    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerStarted","Data":"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"}
Oct 30 18:50:53.571688 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.571574    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:53.571790 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.571722    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:53.572951 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.572924    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:50:53.617727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:53.617670    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podStartSLOduration=2.76198549 podStartE2EDuration="6.617654409s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.855668919s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:53.616182799 +0000 UTC m=+815.376191273" watchObservedRunningTime="2026-10-30 18:50:53.617654409 +0000 UTC m=+815.377662872"
Oct 30 18:50:54.575705 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:54.575661    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:50:57.229426 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:57.229374    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:50:57.229804 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:57.229520    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:50:59.580535 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:59.580506    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:50:59.581012 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:50:59.580964    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:02.229427 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:02.229377    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:51:02.550271 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:02.550167    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:51:02.888230 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:02.888194    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:03.083107 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.083071    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:03.282762 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.282678    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:51:03.283163 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.283018    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" containerID="cri-o://daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7" gracePeriod=30
Oct 30 18:51:03.283163 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.283094    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" containerID="cri-o://b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210" gracePeriod=30
Oct 30 18:51:03.283296 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.283094    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" containerID="cri-o://b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8" gracePeriod=30
Oct 30 18:51:03.363183 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.363151    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:03.616128 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.616090    4938 generic.go:356] "Generic (PLEG): container finished" podID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerID="b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210" exitCode=0
Oct 30 18:51:03.616353 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:03.616167    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerDied","Data":"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"}
Oct 30 18:51:04.239938 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:04.239896    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:07.229812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:07.229765    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:51:09.240214 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:09.240098    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:09.581124 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:09.581083    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:09.641833 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:09.641796    4938 generic.go:356] "Generic (PLEG): container finished" podID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerID="daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7" exitCode=0
Oct 30 18:51:09.642020 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:09.641866    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerDied","Data":"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"}
Oct 30 18:51:10.032551 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:10.032459    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:11.389032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:11.388980    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:51:11.389483 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:11.389337    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:12.230039 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:12.229992    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Oct 30 18:51:13.661011 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.660974    4938 generic.go:356] "Generic (PLEG): container finished" podID="8be248a9-64d7-4f5c-b128-838653806aff" containerID="cc3f5cb5858bac1e249c44c90896be464704abb3505a592ae04733c28eb0c207" exitCode=137
Oct 30 18:51:13.661476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.661055    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerDied","Data":"cc3f5cb5858bac1e249c44c90896be464704abb3505a592ae04733c28eb0c207"}
Oct 30 18:51:13.816778 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.816751    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.895570    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\") pod \"8be248a9-64d7-4f5c-b128-838653806aff\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") "
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.895635    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\") pod \"8be248a9-64d7-4f5c-b128-838653806aff\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") "
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.895680    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\") pod \"8be248a9-64d7-4f5c-b128-838653806aff\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") "
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.895735    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\" (UniqueName: \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\") pod \"8be248a9-64d7-4f5c-b128-838653806aff\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") "
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.895808    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\") pod \"8be248a9-64d7-4f5c-b128-838653806aff\" (UID: \"8be248a9-64d7-4f5c-b128-838653806aff\") "
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.896662    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config" pod "8be248a9-64d7-4f5c-b128-838653806aff" (UID: "8be248a9-64d7-4f5c-b128-838653806aff"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.896671    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert" pod "8be248a9-64d7-4f5c-b128-838653806aff" (UID: "8be248a9-64d7-4f5c-b128-838653806aff"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:13.898274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.896882    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location" pod "8be248a9-64d7-4f5c-b128-838653806aff" (UID: "8be248a9-64d7-4f5c-b128-838653806aff"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:51:13.902699 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.902646    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls" pod "8be248a9-64d7-4f5c-b128-838653806aff" (UID: "8be248a9-64d7-4f5c-b128-838653806aff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:51:13.902923 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.902900    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv" pod "8be248a9-64d7-4f5c-b128-838653806aff" (UID: "8be248a9-64d7-4f5c-b128-838653806aff"). InnerVolumeSpecName "kube-api-access-qpvbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:13.996955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.996851    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8be248a9-64d7-4f5c-b128-838653806aff-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:13.996955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.996886    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8be248a9-64d7-4f5c-b128-838653806aff-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:13.996955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.996901    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:13.996955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.996913    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8be248a9-64d7-4f5c-b128-838653806aff-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:13.996955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:13.996925    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpvbv\" (UniqueName: \"kubernetes.io/projected/8be248a9-64d7-4f5c-b128-838653806aff-kube-api-access-qpvbv\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:14.240092 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.240046    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:14.240267 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.240219    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:51:14.667047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.666963    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg" event={"ID":"8be248a9-64d7-4f5c-b128-838653806aff","Type":"ContainerDied","Data":"ec1b0e61c320cc6ef611a89a606f47411801569a69edbf38aeb5b447d42e7e78"}
Oct 30 18:51:14.667047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.666983    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"
Oct 30 18:51:14.667047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.667032    4938 scope.go:122] "RemoveContainer" containerID="74c4a7abef9df050b1764b56913287a0ec570a4e8bdfb2cc668dd828161659f1"
Oct 30 18:51:14.675266 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.675228    4938 scope.go:122] "RemoveContainer" containerID="cc3f5cb5858bac1e249c44c90896be464704abb3505a592ae04733c28eb0c207"
Oct 30 18:51:14.683452 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.683421    4938 scope.go:122] "RemoveContainer" containerID="df088432d2f9fe6ed3f230d8dfc436217bfb0cbe4ea9393d30bc59e509a95657"
Oct 30 18:51:14.720768 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.720726    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:51:14.733816 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:14.733779    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-vzgpg"]
Oct 30 18:51:16.392755 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:16.392722    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8be248a9-64d7-4f5c-b128-838653806aff" path="/var/lib/kubelet/pods/8be248a9-64d7-4f5c-b128-838653806aff/volumes"
Oct 30 18:51:19.239762 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:19.239710    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:19.581990 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:19.581951    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:21.388753 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:21.388703    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:51:21.389129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:21.389028    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:24.240195 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:24.240151    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:29.240553 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:29.240508    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Oct 30 18:51:29.581302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:29.581241    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:31.388354 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:31.388263    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Oct 30 18:51:31.388927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:31.388391    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:51:31.388927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:31.388657    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:31.388927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:31.388787    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:51:32.549452 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:32.549401    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:51:33.440045 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.440019    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:51:33.559634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.559529    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\") pod \"2ce9b27a-af0d-4408-b985-5d2af7061d66\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") "
Oct 30 18:51:33.559634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.559601    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\" (UniqueName: \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\") pod \"2ce9b27a-af0d-4408-b985-5d2af7061d66\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") "
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.559643    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\") pod \"2ce9b27a-af0d-4408-b985-5d2af7061d66\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") "
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.559675    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\") pod \"2ce9b27a-af0d-4408-b985-5d2af7061d66\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") "
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.559709    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\") pod \"2ce9b27a-af0d-4408-b985-5d2af7061d66\" (UID: \"2ce9b27a-af0d-4408-b985-5d2af7061d66\") "
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.560007    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location" pod "2ce9b27a-af0d-4408-b985-5d2af7061d66" (UID: "2ce9b27a-af0d-4408-b985-5d2af7061d66"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.560092    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert" pod "2ce9b27a-af0d-4408-b985-5d2af7061d66" (UID: "2ce9b27a-af0d-4408-b985-5d2af7061d66"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:33.560206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.560185    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config" pod "2ce9b27a-af0d-4408-b985-5d2af7061d66" (UID: "2ce9b27a-af0d-4408-b985-5d2af7061d66"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:51:33.561898 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.561868    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz" pod "2ce9b27a-af0d-4408-b985-5d2af7061d66" (UID: "2ce9b27a-af0d-4408-b985-5d2af7061d66"). InnerVolumeSpecName "kube-api-access-7vdqz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:51:33.562003 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.561987    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls" pod "2ce9b27a-af0d-4408-b985-5d2af7061d66" (UID: "2ce9b27a-af0d-4408-b985-5d2af7061d66"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:51:33.661297 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.661238    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2ce9b27a-af0d-4408-b985-5d2af7061d66-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:33.661297 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.661292    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vdqz\" (UniqueName: \"kubernetes.io/projected/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-api-access-7vdqz\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:33.661297 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.661303    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2ce9b27a-af0d-4408-b985-5d2af7061d66-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:33.661513 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.661312    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:33.661513 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.661320    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2ce9b27a-af0d-4408-b985-5d2af7061d66-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:51:33.760982 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.760944    4938 generic.go:356] "Generic (PLEG): container finished" podID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerID="b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8" exitCode=0
Oct 30 18:51:33.761172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.761007    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"
Oct 30 18:51:33.761172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.761025    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerDied","Data":"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"}
Oct 30 18:51:33.761172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.761111    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9" event={"ID":"2ce9b27a-af0d-4408-b985-5d2af7061d66","Type":"ContainerDied","Data":"15e3c66c1085c11817c1aa6e509d151faf84e934ea2d0298d55138ca008af87e"}
Oct 30 18:51:33.761172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.761153    4938 scope.go:122] "RemoveContainer" containerID="b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"
Oct 30 18:51:33.770411 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.770393    4938 scope.go:122] "RemoveContainer" containerID="b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"
Oct 30 18:51:33.777979 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.777956    4938 scope.go:122] "RemoveContainer" containerID="daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"
Oct 30 18:51:33.785387 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.785364    4938 scope.go:122] "RemoveContainer" containerID="26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"
Oct 30 18:51:33.792861 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.792833    4938 scope.go:122] "RemoveContainer" containerID="b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"
Oct 30 18:51:33.793140 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:33.793121    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8\": container with ID starting with b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8 not found: ID does not exist" containerID="b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"
Oct 30 18:51:33.793196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793151    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8"} err="failed to get container status \"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8\": rpc error: code = NotFound desc = could not find container \"b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8\": container with ID starting with b1b08610de4387464a471b90d7afe78aa805f76876eae8074b392e64124c9bc8 not found: ID does not exist"
Oct 30 18:51:33.793196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793171    4938 scope.go:122] "RemoveContainer" containerID="b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"
Oct 30 18:51:33.793481 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:33.793458    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210\": container with ID starting with b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210 not found: ID does not exist" containerID="b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"
Oct 30 18:51:33.793541 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793492    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210"} err="failed to get container status \"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210\": rpc error: code = NotFound desc = could not find container \"b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210\": container with ID starting with b2b7a6d568ba2e5ea34fcc8c9fdc0de380c14063e5f36b7edb14fcd58473c210 not found: ID does not exist"
Oct 30 18:51:33.793541 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793511    4938 scope.go:122] "RemoveContainer" containerID="daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"
Oct 30 18:51:33.793807 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:33.793789    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7\": container with ID starting with daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7 not found: ID does not exist" containerID="daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"
Oct 30 18:51:33.793873 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793812    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7"} err="failed to get container status \"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7\": rpc error: code = NotFound desc = could not find container \"daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7\": container with ID starting with daf5f53241088071a269ecbee2a719c885b643079ebe7dfc4ab7ffbc3e00c1b7 not found: ID does not exist"
Oct 30 18:51:33.793873 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.793826    4938 scope.go:122] "RemoveContainer" containerID="26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"
Oct 30 18:51:33.794065 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:51:33.794047    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51\": container with ID starting with 26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51 not found: ID does not exist" containerID="26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"
Oct 30 18:51:33.794122 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.794070    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51"} err="failed to get container status \"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51\": rpc error: code = NotFound desc = could not find container \"26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51\": container with ID starting with 26f8f7b16a6bf46ab0d379462b54b1eda88f97336ba54b40dda552410db36f51 not found: ID does not exist"
Oct 30 18:51:33.818278 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.818182    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:51:33.823273 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:33.823236    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-raw-sklearn-batcher-cae97-predictor-686b6b6cb6-njtf9"]
Oct 30 18:51:34.393598 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:34.393562    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" path="/var/lib/kubelet/pods/2ce9b27a-af0d-4408-b985-5d2af7061d66/volumes"
Oct 30 18:51:35.194033 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.193995    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.281366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281329    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:51:35.281670 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281655    4938 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281672    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281684    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281689    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281696    4938 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281701    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281708    4938 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="storage-initializer"
Oct 30 18:51:35.281720 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281713    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="storage-initializer"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281733    4938 cpu_manager.go:414] "containerMap: removing container" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281739    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281749    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281755    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281766    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="storage-initializer"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281771    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="storage-initializer"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281818    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="agent"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281827    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kserve-container"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281836    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kserve-container"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281842    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="8be248a9-64d7-4f5c-b128-838653806aff" containerName="kube-rbac-proxy"
Oct 30 18:51:35.281918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.281846    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ce9b27a-af0d-4408-b985-5d2af7061d66" containerName="kube-rbac-proxy"
Oct 30 18:51:35.284902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.284878    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.305453 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.305422    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 18:51:35.305453 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.305438    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:51:35.305453 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.305432    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-graph-raw-14d1b-predictor-serving-cert\""
Oct 30 18:51:35.318736 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.318702    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 18:51:35.318736 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.318704    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-graph-raw-14d1b-kube-rbac-proxy-sar-config\""
Oct 30 18:51:35.318991 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.318713    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 18:51:35.318991 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.318713    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 18:51:35.319963 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.319936    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:51:35.447053 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.446969    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.473691 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.473650    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.473878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.473700    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvmr9\" (UniqueName: \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.473878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.473722    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.473878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.473869    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.473998 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.473914    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.574667 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.574620    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.574871 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.574692    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.574871 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.574753    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvmr9\" (UniqueName: \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.574871 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.574776    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.574871 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.574830    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.575346 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.575323    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.575474 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.575454    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.575516 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.575496    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.577213 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.577188    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.594639 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.594609    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.595416 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.595390    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvmr9\" (UniqueName: \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\") pod \"isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.601130 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.601110    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:35.642109 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.642067    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:51:35.646373 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.646345    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.662114 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.662021    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-xgboost-graph-raw-14d1b-kube-rbac-proxy-sar-config\""
Oct 30 18:51:35.662114 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.662072    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-xgboost-graph-raw-14d1b-predictor-serving-cert\""
Oct 30 18:51:35.667583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.667558    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:51:35.770927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.770837    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:35.776553 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.776529    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.776652 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.776575    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.776701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.776664    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twwfq\" (UniqueName: \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.776744 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.776723    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.776841 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.776825    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.877528 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.877486    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.877727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.877536    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.877727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.877575    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.877727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.877632    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.877727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.877674    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twwfq\" (UniqueName: \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.878054 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.878030    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.878318 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.878300    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.878370 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.878344    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.880185 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.880161    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.897869 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.897833    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twwfq\" (UniqueName: \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\") pod \"isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") " pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:35.914687 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.914653    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:51:35.917748 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:51:35.917721    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5caa187_a5c7_40fb_967d_db848b82d1c5.slice/crio-b5cd6a13f8b0786c11520493d041362cfc9f6db5c6989c3ccad8bcc4d2373ee4 WatchSource:0}: Error finding container b5cd6a13f8b0786c11520493d041362cfc9f6db5c6989c3ccad8bcc4d2373ee4: Status 404 returned error can't find the container with id b5cd6a13f8b0786c11520493d041362cfc9f6db5c6989c3ccad8bcc4d2373ee4
Oct 30 18:51:35.969751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:35.969731    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:51:36.280335 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:36.280224    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:51:36.282878 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:51:36.282848    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad521ea_552b_4980_b523_0c59eaf9e505.slice/crio-04a797e4dc4fd8ece6c5dcb7430935e6f5208aeee92b7525192428da7d5efcd2 WatchSource:0}: Error finding container 04a797e4dc4fd8ece6c5dcb7430935e6f5208aeee92b7525192428da7d5efcd2: Status 404 returned error can't find the container with id 04a797e4dc4fd8ece6c5dcb7430935e6f5208aeee92b7525192428da7d5efcd2
Oct 30 18:51:36.776785 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:36.776729    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerStarted","Data":"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"}
Oct 30 18:51:36.776982 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:36.776809    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerStarted","Data":"b5cd6a13f8b0786c11520493d041362cfc9f6db5c6989c3ccad8bcc4d2373ee4"}
Oct 30 18:51:36.778373 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:36.778340    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerStarted","Data":"35dd1c2a6b565d504cb04302da339fabc71d952d7a1801605c4c5437fc671c02"}
Oct 30 18:51:36.778494 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:36.778399    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerStarted","Data":"04a797e4dc4fd8ece6c5dcb7430935e6f5208aeee92b7525192428da7d5efcd2"}
Oct 30 18:51:39.581557 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:39.581518    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:39.792510 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:39.792473    4938 generic.go:356] "Generic (PLEG): container finished" podID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerID="947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851" exitCode=0
Oct 30 18:51:39.792683 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:39.792552    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerDied","Data":"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"}
Oct 30 18:51:40.799043 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.799006    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerStarted","Data":"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"}
Oct 30 18:51:40.799515 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.799081    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerStarted","Data":"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"}
Oct 30 18:51:40.799515 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.799300    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:40.800675 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.800649    4938 generic.go:356] "Generic (PLEG): container finished" podID="cad521ea-552b-4980-b523-0c59eaf9e505" containerID="35dd1c2a6b565d504cb04302da339fabc71d952d7a1801605c4c5437fc671c02" exitCode=0
Oct 30 18:51:40.800806 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.800680    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerDied","Data":"35dd1c2a6b565d504cb04302da339fabc71d952d7a1801605c4c5437fc671c02"}
Oct 30 18:51:40.802123 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.802109    4938 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:51:40.871329 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:40.871272    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podStartSLOduration=2.114655879 podStartE2EDuration="5.871241346s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.756585467s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:40.857772845 +0000 UTC m=+862.617781309" watchObservedRunningTime="2026-10-30 18:51:40.871241346 +0000 UTC m=+862.631249808"
Oct 30 18:51:41.808598 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:41.808486    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:41.809887 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:41.809849    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:51:42.814812 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:42.814412    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:51:47.819264 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:47.819222    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:51:47.819930 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:47.819901    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:51:49.581626 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:49.581584    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:51:57.819856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:57.819800    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:51:59.581058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:51:59.580997    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:52:00.903891 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:00.903847    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerStarted","Data":"11187deb4c2993877b43b40f46d5fbf4182fdc1635cbf148f18fb25bac62c707"}
Oct 30 18:52:00.904293 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:00.903919    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerStarted","Data":"691a66feab1de9a6196dd70a8bd441a1039d2da262822499533ad979ce4a1f33"}
Oct 30 18:52:00.904293 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:00.904124    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:52:01.910110 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:01.910073    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:52:01.912457 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:01.912421    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:02.549655 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:52:02.549613    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:52:02.913573 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:02.913522    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:07.820426 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:07.820386    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:07.918128 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:07.918100    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:52:07.918603 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:07.918577    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:07.976846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:07.976785    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podStartSLOduration=9.824048979 podStartE2EDuration="32.976769812s" totalImagesPullingTime="19.441055558s" totalInitContainerRuntime="3.711665275s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:00.990646269 +0000 UTC m=+882.750654731" watchObservedRunningTime="2026-10-30 18:52:07.976769812 +0000 UTC m=+889.736778271"
Oct 30 18:52:09.581821 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:09.581782    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:52:10.106516 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:10.106479    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:16.388650 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:16.388601    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Oct 30 18:52:17.820490 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:17.820446    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:17.919147 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:17.919101    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:18.469399 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:18.469367    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:52:18.471433 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:18.471406    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:52:18.473467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:18.473445    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:52:18.475891 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:18.475870    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:52:26.392828 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:26.392799    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:52:27.820114 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:27.820045    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:27.919021 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:27.918975    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:28.117515 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:28.117417    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:52:28.117667 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:28.117649    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container" containerID="cri-o://12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441" gracePeriod=30
Oct 30 18:52:28.117733 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:28.117689    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kube-rbac-proxy" containerID="cri-o://aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee" gracePeriod=30
Oct 30 18:52:28.198887 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:28.198849    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:29.021460 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:29.021422    4938 generic.go:356] "Generic (PLEG): container finished" podID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerID="aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee" exitCode=0
Oct 30 18:52:29.021838 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:29.021482    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerDied","Data":"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"}
Oct 30 18:52:29.576273 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:29.576207    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Oct 30 18:52:32.548170 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:52:32.548131    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:52:33.662583 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.662552    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:52:33.766415 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766323    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\" (UniqueName: \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\") pod \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") "
Oct 30 18:52:33.766415 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766397    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\" (UniqueName: \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\") pod \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") "
Oct 30 18:52:33.766615 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766461    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\") pod \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") "
Oct 30 18:52:33.766615 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766588    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\") pod \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") "
Oct 30 18:52:33.766719 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766625    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\") pod \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\" (UID: \"069cd92b-8442-4f41-bb9b-9dae8c94fdec\") "
Oct 30 18:52:33.766941 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766803    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config" pod "069cd92b-8442-4f41-bb9b-9dae8c94fdec" (UID: "069cd92b-8442-4f41-bb9b-9dae8c94fdec"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.767054 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.766997    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert" pod "069cd92b-8442-4f41-bb9b-9dae8c94fdec" (UID: "069cd92b-8442-4f41-bb9b-9dae8c94fdec"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:52:33.767054 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.767006    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location" pod "069cd92b-8442-4f41-bb9b-9dae8c94fdec" (UID: "069cd92b-8442-4f41-bb9b-9dae8c94fdec"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:33.768594 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.768565    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72" pod "069cd92b-8442-4f41-bb9b-9dae8c94fdec" (UID: "069cd92b-8442-4f41-bb9b-9dae8c94fdec"). InnerVolumeSpecName "kube-api-access-2nd72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:33.768681 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.768638    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls" pod "069cd92b-8442-4f41-bb9b-9dae8c94fdec" (UID: "069cd92b-8442-4f41-bb9b-9dae8c94fdec"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:52:33.867742 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.867707    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.867742 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.867736    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/069cd92b-8442-4f41-bb9b-9dae8c94fdec-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.867742 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.867746    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.867984 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.867756    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/069cd92b-8442-4f41-bb9b-9dae8c94fdec-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:52:33.867984 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:33.867765    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2nd72\" (UniqueName: \"kubernetes.io/projected/069cd92b-8442-4f41-bb9b-9dae8c94fdec-kube-api-access-2nd72\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:52:34.044595 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.044508    4938 generic.go:356] "Generic (PLEG): container finished" podID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerID="12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441" exitCode=0
Oct 30 18:52:34.044595 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.044565    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerDied","Data":"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"}
Oct 30 18:52:34.044595 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.044585    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"
Oct 30 18:52:34.044835 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.044639    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t" event={"ID":"069cd92b-8442-4f41-bb9b-9dae8c94fdec","Type":"ContainerDied","Data":"ecaa3de286a7a57915dfc10ddc6e0eace2ad607e4ef6f4799921affa0f899187"}
Oct 30 18:52:34.044835 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.044666    4938 scope.go:122] "RemoveContainer" containerID="aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"
Oct 30 18:52:34.053587 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.053560    4938 scope.go:122] "RemoveContainer" containerID="12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"
Oct 30 18:52:34.063482 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.063451    4938 scope.go:122] "RemoveContainer" containerID="f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"
Oct 30 18:52:34.071416 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.071380    4938 scope.go:122] "RemoveContainer" containerID="aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"
Oct 30 18:52:34.071685 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:52:34.071664    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee\": container with ID starting with aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee not found: ID does not exist" containerID="aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"
Oct 30 18:52:34.071765 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.071699    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee"} err="failed to get container status \"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee\": rpc error: code = NotFound desc = could not find container \"aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee\": container with ID starting with aa75f4ed37e69566d0e992f197b3b210abeaca34847f49d09f6d67cce07a43ee not found: ID does not exist"
Oct 30 18:52:34.071765 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.071730    4938 scope.go:122] "RemoveContainer" containerID="12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"
Oct 30 18:52:34.071994 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:52:34.071973    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441\": container with ID starting with 12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441 not found: ID does not exist" containerID="12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"
Oct 30 18:52:34.072038 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.072000    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441"} err="failed to get container status \"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441\": rpc error: code = NotFound desc = could not find container \"12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441\": container with ID starting with 12efe565d903de555eb5edee03c05327773d2a888fee559218e2561ed5f51441 not found: ID does not exist"
Oct 30 18:52:34.072038 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.072016    4938 scope.go:122] "RemoveContainer" containerID="f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"
Oct 30 18:52:34.072316 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:52:34.072297    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd\": container with ID starting with f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd not found: ID does not exist" containerID="f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"
Oct 30 18:52:34.072383 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.072325    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd"} err="failed to get container status \"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd\": rpc error: code = NotFound desc = could not find container \"f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd\": container with ID starting with f8f231c9382cc2c367b09332b028393da4c88479e863a3870e035f6a1906b7fd not found: ID does not exist"
Oct 30 18:52:34.099488 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.099456    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:52:34.104470 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.104441    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-rollback-predictor-95869db44-dpk2t"]
Oct 30 18:52:34.393105 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:34.393066    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" path="/var/lib/kubelet/pods/069cd92b-8442-4f41-bb9b-9dae8c94fdec/volumes"
Oct 30 18:52:35.750030 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.749993    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:35.848827 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.848790    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:52:35.849147 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849131    4938 cpu_manager.go:414] "containerMap: removing container" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="storage-initializer"
Oct 30 18:52:35.849211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849150    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="storage-initializer"
Oct 30 18:52:35.849211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849164    4938 cpu_manager.go:414] "containerMap: removing container" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container"
Oct 30 18:52:35.849211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849169    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container"
Oct 30 18:52:35.849211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849184    4938 cpu_manager.go:414] "containerMap: removing container" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kube-rbac-proxy"
Oct 30 18:52:35.849211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849189    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kube-rbac-proxy"
Oct 30 18:52:35.849391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849264    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kserve-container"
Oct 30 18:52:35.849391 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.849274    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="069cd92b-8442-4f41-bb9b-9dae8c94fdec" containerName="kube-rbac-proxy"
Oct 30 18:52:35.852382 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.852358    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:35.870932 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.870904    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:52:35.883597 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883566    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 18:52:35.883818 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883796    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\""
Oct 30 18:52:35.883902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883810    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"seaweedfs-s3-creds\""
Oct 30 18:52:35.883902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883831    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 18:52:35.883902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883861    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"default-dockercfg-gh4qc\""
Oct 30 18:52:35.883902 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.883902    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-canary-stop-v2-predictor-serving-cert\""
Oct 30 18:52:35.885214 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.885190    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:52:35.907503 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.907469    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:35.984904 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.984870    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:35.985100 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.984923    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26zp\" (UniqueName: \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:35.985100 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.985009    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:35.985100 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.985053    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:35.985100 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:35.985080    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.005564 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.005473    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:52:36.009114 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.009095    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.025064 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.025035    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-canary-stop-predictor-serving-cert\""
Oct 30 18:52:36.031314 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.031285    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:52:36.045783 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.045748    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:36.085953 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.085913    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.085953 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.085953    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086200 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.085986    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086200 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.086021    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26zp\" (UniqueName: \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086200 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.086050    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086552 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.086525    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086727 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.086709    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.086793 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.086753    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.088547 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.088529    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.106759 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.106726    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26zp\" (UniqueName: \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\") pod \"isvc-canary-stop-v2-predictor-784d7544fc-p74zg\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.172634 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.172599    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:36.187081 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.187048    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.187211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.187091    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw74r\" (UniqueName: \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.187211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.187167    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.187211 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.187194    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.187364 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.187239    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.287751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.287650    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.287751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.287691    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw74r\" (UniqueName: \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.287751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.287714    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.287751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.287733    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.288101 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.287760    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.288263 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.288227    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.288497 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.288475    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.288645 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.288626    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.290154 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.290136    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.307729 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.307699    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw74r\" (UniqueName: \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\") pod \"isvc-canary-stop-predictor-77b4bb9d84-7s98g\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") " pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.325473 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.325434    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:36.483284 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.483242    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:52:36.485863 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:52:36.485836    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0abd299c_cd4d_4328_ac8f_f0267848a4c5.slice/crio-20881f931100a643cc45855b469c5f2f8ade299bf18793486ba66a50e02e2781 WatchSource:0}: Error finding container 20881f931100a643cc45855b469c5f2f8ade299bf18793486ba66a50e02e2781: Status 404 returned error can't find the container with id 20881f931100a643cc45855b469c5f2f8ade299bf18793486ba66a50e02e2781
Oct 30 18:52:36.627461 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:36.627431    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:52:36.630750 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:52:36.630718    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc458501d_080b_40b6_bfbd_b534f55fa813.slice/crio-2a8eebec07b0206204a9f0bd057847419c6d1f105c93c959a11dab52924bd051 WatchSource:0}: Error finding container 2a8eebec07b0206204a9f0bd057847419c6d1f105c93c959a11dab52924bd051: Status 404 returned error can't find the container with id 2a8eebec07b0206204a9f0bd057847419c6d1f105c93c959a11dab52924bd051
Oct 30 18:52:37.061271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.060449    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerStarted","Data":"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"}
Oct 30 18:52:37.061271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.060521    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerStarted","Data":"2a8eebec07b0206204a9f0bd057847419c6d1f105c93c959a11dab52924bd051"}
Oct 30 18:52:37.062581 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.062547    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerStarted","Data":"93809762dd67d90c55d5653749ab5e32f179bfb308a0475f92d3be0116ea20c2"}
Oct 30 18:52:37.062713 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.062608    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerStarted","Data":"20881f931100a643cc45855b469c5f2f8ade299bf18793486ba66a50e02e2781"}
Oct 30 18:52:37.820403 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.820347    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:37.918563 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:37.918519    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:41.081970 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:41.081931    4938 generic.go:356] "Generic (PLEG): container finished" podID="c458501d-080b-40b6-bfbd-b534f55fa813" containerID="289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69" exitCode=0
Oct 30 18:52:41.082413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:41.082007    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerDied","Data":"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"}
Oct 30 18:52:41.083639 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:41.083613    4938 generic.go:356] "Generic (PLEG): container finished" podID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerID="93809762dd67d90c55d5653749ab5e32f179bfb308a0475f92d3be0116ea20c2" exitCode=0
Oct 30 18:52:41.083752 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:41.083682    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerDied","Data":"93809762dd67d90c55d5653749ab5e32f179bfb308a0475f92d3be0116ea20c2"}
Oct 30 18:52:42.097666 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.097622    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerStarted","Data":"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"}
Oct 30 18:52:42.098161 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.097732    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerStarted","Data":"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"}
Oct 30 18:52:42.098161 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.098010    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:42.099855 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.099823    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerStarted","Data":"0f2a87cdff6268c43ab7ea70cba023b00f8d92a80a3da9c0aa01c075771c1834"}
Oct 30 18:52:42.099966 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.099884    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerStarted","Data":"8523c4edc7c2f12b2fcada92d465096eacdadf9b99f1d858b193bdca6d015c7f"}
Oct 30 18:52:42.100096 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.100079    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:42.157299 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.157207    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podStartSLOduration=3.408811586 podStartE2EDuration="7.157187004s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.748375418s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:42.15512378 +0000 UTC m=+923.915132268" watchObservedRunningTime="2026-10-30 18:52:42.157187004 +0000 UTC m=+923.917195470"
Oct 30 18:52:42.199946 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:42.199888    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podStartSLOduration=3.345965589 podStartE2EDuration="7.199877377s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.853911788s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:42.19850371 +0000 UTC m=+923.958512172" watchObservedRunningTime="2026-10-30 18:52:42.199877377 +0000 UTC m=+923.959885837"
Oct 30 18:52:43.104123 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:43.104082    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:43.104649 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:43.104152    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:43.108799 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:43.108765    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:52:43.108799 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:43.108782    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:52:44.108818 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:44.108773    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:52:44.109208 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:44.108864    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:52:47.820222 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:47.820181    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:47.918843 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:47.918799    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:49.115048 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:49.115016    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:52:49.115550 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:49.115460    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:52:49.115603 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:49.115559    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:52:49.116041 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:49.116016    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:52:49.387771 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:49.387670    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:52:57.918978 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:57.918937    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:52:59.116201 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:59.116159    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:52:59.116630 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:59.116159    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:52:59.387755 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:52:59.387665    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:53:02.550626 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:53:02.550543    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:53:07.918747 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:07.918707    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:53:09.116411 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:09.116368    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:09.116817 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:09.116370    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:09.388459 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:09.388379    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:53:10.263500 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:10.263469    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:16.392886 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:16.392858    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:53:19.115576 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:19.115532    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:19.116003 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:19.115826    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:25.477962 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:25.477925    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:25.621108 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:25.621071    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:28.733327 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:28.733289    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:29.115933 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:29.115895    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:29.116121 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:29.115895    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:32.550541 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:53:32.550505    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:53:35.878434 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.878397    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:53:35.878899 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.878629    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" containerID="cri-o://50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2" gracePeriod=30
Oct 30 18:53:35.878899 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.878705    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kube-rbac-proxy" containerID="cri-o://a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081" gracePeriod=30
Oct 30 18:53:35.961005 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.960966    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:35.975286 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.975239    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:53:35.975505 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.975475    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" containerID="cri-o://691a66feab1de9a6196dd70a8bd441a1039d2da262822499533ad979ce4a1f33" gracePeriod=30
Oct 30 18:53:35.975632 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:35.975525    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kube-rbac-proxy" containerID="cri-o://11187deb4c2993877b43b40f46d5fbf4182fdc1635cbf148f18fb25bac62c707" gracePeriod=30
Oct 30 18:53:36.109984 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.109953    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:36.343979 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.343932    4938 generic.go:356] "Generic (PLEG): container finished" podID="cad521ea-552b-4980-b523-0c59eaf9e505" containerID="11187deb4c2993877b43b40f46d5fbf4182fdc1635cbf148f18fb25bac62c707" exitCode=0
Oct 30 18:53:36.344165 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.344002    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerDied","Data":"11187deb4c2993877b43b40f46d5fbf4182fdc1635cbf148f18fb25bac62c707"}
Oct 30 18:53:36.346261 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.346228    4938 generic.go:356] "Generic (PLEG): container finished" podID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerID="a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081" exitCode=0
Oct 30 18:53:36.346383 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.346271    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerDied","Data":"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"}
Oct 30 18:53:36.388856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:36.388808    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused"
Oct 30 18:53:37.814799 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:37.814754    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.29:8643/healthz\": dial tcp 10.133.0.29:8643: connect: connection refused"
Oct 30 18:53:37.914134 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:37.914087    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused"
Oct 30 18:53:39.115833 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:39.115789    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:39.116377 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:39.115928    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:39.388484 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:39.388403    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.29:8080: connect: connection refused"
Oct 30 18:53:40.369216 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.369178    4938 generic.go:356] "Generic (PLEG): container finished" podID="cad521ea-552b-4980-b523-0c59eaf9e505" containerID="691a66feab1de9a6196dd70a8bd441a1039d2da262822499533ad979ce4a1f33" exitCode=0
Oct 30 18:53:40.369622 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.369268    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerDied","Data":"691a66feab1de9a6196dd70a8bd441a1039d2da262822499533ad979ce4a1f33"}
Oct 30 18:53:40.418146 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.418121    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:53:40.524610 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.524519    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\" (UniqueName: \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\") pod \"cad521ea-552b-4980-b523-0c59eaf9e505\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") "
Oct 30 18:53:40.524610 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.524563    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\") pod \"cad521ea-552b-4980-b523-0c59eaf9e505\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") "
Oct 30 18:53:40.524846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.524630    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\") pod \"cad521ea-552b-4980-b523-0c59eaf9e505\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") "
Oct 30 18:53:40.524846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.524663    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\" (UniqueName: \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\") pod \"cad521ea-552b-4980-b523-0c59eaf9e505\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") "
Oct 30 18:53:40.524846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.524695    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\") pod \"cad521ea-552b-4980-b523-0c59eaf9e505\" (UID: \"cad521ea-552b-4980-b523-0c59eaf9e505\") "
Oct 30 18:53:40.525065 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.525030    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert" pod "cad521ea-552b-4980-b523-0c59eaf9e505" (UID: "cad521ea-552b-4980-b523-0c59eaf9e505"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:40.525065 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.525043    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config" pod "cad521ea-552b-4980-b523-0c59eaf9e505" (UID: "cad521ea-552b-4980-b523-0c59eaf9e505"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:40.525162 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.525122    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location" pod "cad521ea-552b-4980-b523-0c59eaf9e505" (UID: "cad521ea-552b-4980-b523-0c59eaf9e505"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:40.526738 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.526715    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls" pod "cad521ea-552b-4980-b523-0c59eaf9e505" (UID: "cad521ea-552b-4980-b523-0c59eaf9e505"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:40.526827 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.526801    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq" pod "cad521ea-552b-4980-b523-0c59eaf9e505" (UID: "cad521ea-552b-4980-b523-0c59eaf9e505"). InnerVolumeSpecName "kube-api-access-twwfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:40.625619 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.625577    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:40.625619 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.625612    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twwfq\" (UniqueName: \"kubernetes.io/projected/cad521ea-552b-4980-b523-0c59eaf9e505-kube-api-access-twwfq\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:40.625619 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.625623    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/cad521ea-552b-4980-b523-0c59eaf9e505-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:40.625856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.625636    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cad521ea-552b-4980-b523-0c59eaf9e505-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:40.625856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:40.625646    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/cad521ea-552b-4980-b523-0c59eaf9e505-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.223529 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.223504    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:53:41.332001 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.331959    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\") pod \"c5caa187-a5c7-40fb-967d-db848b82d1c5\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") "
Oct 30 18:53:41.332206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332015    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\" (UniqueName: \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\") pod \"c5caa187-a5c7-40fb-967d-db848b82d1c5\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") "
Oct 30 18:53:41.332206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332080    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\") pod \"c5caa187-a5c7-40fb-967d-db848b82d1c5\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") "
Oct 30 18:53:41.332206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332114    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\") pod \"c5caa187-a5c7-40fb-967d-db848b82d1c5\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") "
Oct 30 18:53:41.332206 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332151    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\") pod \"c5caa187-a5c7-40fb-967d-db848b82d1c5\" (UID: \"c5caa187-a5c7-40fb-967d-db848b82d1c5\") "
Oct 30 18:53:41.332527 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332498    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location" pod "c5caa187-a5c7-40fb-967d-db848b82d1c5" (UID: "c5caa187-a5c7-40fb-967d-db848b82d1c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:53:41.332593 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332498    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert" pod "c5caa187-a5c7-40fb-967d-db848b82d1c5" (UID: "c5caa187-a5c7-40fb-967d-db848b82d1c5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:41.332661 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.332635    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config" pod "c5caa187-a5c7-40fb-967d-db848b82d1c5" (UID: "c5caa187-a5c7-40fb-967d-db848b82d1c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:53:41.334189 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.334157    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls" pod "c5caa187-a5c7-40fb-967d-db848b82d1c5" (UID: "c5caa187-a5c7-40fb-967d-db848b82d1c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:53:41.334336 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.334229    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9" pod "c5caa187-a5c7-40fb-967d-db848b82d1c5" (UID: "c5caa187-a5c7-40fb-967d-db848b82d1c5"). InnerVolumeSpecName "kube-api-access-mvmr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:53:41.375692 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.375655    4938 generic.go:356] "Generic (PLEG): container finished" podID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerID="50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2" exitCode=0
Oct 30 18:53:41.376196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.375725    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"
Oct 30 18:53:41.376196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.375740    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerDied","Data":"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"}
Oct 30 18:53:41.376196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.375812    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6" event={"ID":"c5caa187-a5c7-40fb-967d-db848b82d1c5","Type":"ContainerDied","Data":"b5cd6a13f8b0786c11520493d041362cfc9f6db5c6989c3ccad8bcc4d2373ee4"}
Oct 30 18:53:41.376196 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.375847    4938 scope.go:122] "RemoveContainer" containerID="a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"
Oct 30 18:53:41.377904 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.377883    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk" event={"ID":"cad521ea-552b-4980-b523-0c59eaf9e505","Type":"ContainerDied","Data":"04a797e4dc4fd8ece6c5dcb7430935e6f5208aeee92b7525192428da7d5efcd2"}
Oct 30 18:53:41.378014 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.377958    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"
Oct 30 18:53:41.385594 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.385575    4938 scope.go:122] "RemoveContainer" containerID="50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"
Oct 30 18:53:41.393563 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.393538    4938 scope.go:122] "RemoveContainer" containerID="947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"
Oct 30 18:53:41.401341 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.401314    4938 scope.go:122] "RemoveContainer" containerID="a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"
Oct 30 18:53:41.401675 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:53:41.401651    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081\": container with ID starting with a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081 not found: ID does not exist" containerID="a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"
Oct 30 18:53:41.401772 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.401691    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081"} err="failed to get container status \"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081\": rpc error: code = NotFound desc = could not find container \"a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081\": container with ID starting with a95fe2fd533b0550dca6e20fb1e631f0dc13761a68835a4b619615b192df9081 not found: ID does not exist"
Oct 30 18:53:41.401772 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.401722    4938 scope.go:122] "RemoveContainer" containerID="50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"
Oct 30 18:53:41.401998 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:53:41.401981    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2\": container with ID starting with 50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2 not found: ID does not exist" containerID="50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"
Oct 30 18:53:41.402044 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.402006    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2"} err="failed to get container status \"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2\": rpc error: code = NotFound desc = could not find container \"50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2\": container with ID starting with 50189bb0908381a36d8f853b0bf9a97337ffcdb9c04de83f8a3c23f429ad44c2 not found: ID does not exist"
Oct 30 18:53:41.402044 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.402025    4938 scope.go:122] "RemoveContainer" containerID="947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"
Oct 30 18:53:41.402235 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:53:41.402214    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851\": container with ID starting with 947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851 not found: ID does not exist" containerID="947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"
Oct 30 18:53:41.402366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.402242    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851"} err="failed to get container status \"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851\": rpc error: code = NotFound desc = could not find container \"947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851\": container with ID starting with 947a59db32334f7c5e0054b1faf7bf58531b1ef5a24bbdfc5f2946bdc0547851 not found: ID does not exist"
Oct 30 18:53:41.402366 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.402310    4938 scope.go:122] "RemoveContainer" containerID="11187deb4c2993877b43b40f46d5fbf4182fdc1635cbf148f18fb25bac62c707"
Oct 30 18:53:41.409838 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.409819    4938 scope.go:122] "RemoveContainer" containerID="691a66feab1de9a6196dd70a8bd441a1039d2da262822499533ad979ce4a1f33"
Oct 30 18:53:41.417071 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.417054    4938 scope.go:122] "RemoveContainer" containerID="35dd1c2a6b565d504cb04302da339fabc71d952d7a1801605c4c5437fc671c02"
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.430801    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.434684    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvmr9\" (UniqueName: \"kubernetes.io/projected/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-api-access-mvmr9\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.434721    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c5caa187-a5c7-40fb-967d-db848b82d1c5-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.434735    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c5caa187-a5c7-40fb-967d-db848b82d1c5-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.434750    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.435129 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.434766    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c5caa187-a5c7-40fb-967d-db848b82d1c5-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:53:41.437312 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.437275    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-14d1b-predictor-7fc6f95658-czxg6"]
Oct 30 18:53:41.473526 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.473497    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:53:41.477644 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:41.477617    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-xgboost-graph-raw-14d1b-predictor-667f99c47-9dhmk"]
Oct 30 18:53:42.392203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:42.392167    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" path="/var/lib/kubelet/pods/c5caa187-a5c7-40fb-967d-db848b82d1c5/volumes"
Oct 30 18:53:42.392694 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:42.392679    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" path="/var/lib/kubelet/pods/cad521ea-552b-4980-b523-0c59eaf9e505/volumes"
Oct 30 18:53:43.378590 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.378555    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.477085 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477044    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:53:43.477600 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477543    4938 cpu_manager.go:414] "containerMap: removing container" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="storage-initializer"
Oct 30 18:53:43.477600 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477564    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="storage-initializer"
Oct 30 18:53:43.477600 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477584    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="storage-initializer"
Oct 30 18:53:43.477600 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477592    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="storage-initializer"
Oct 30 18:53:43.477600 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477601    4938 cpu_manager.go:414] "containerMap: removing container" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477608    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477621    4938 cpu_manager.go:414] "containerMap: removing container" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477629    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477649    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477658    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477681    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477689    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477781    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477798    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kserve-container"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477807    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="cad521ea-552b-4980-b523-0c59eaf9e505" containerName="kube-rbac-proxy"
Oct 30 18:53:43.477862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.477816    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5caa187-a5c7-40fb-967d-db848b82d1c5" containerName="kserve-container"
Oct 30 18:53:43.482962 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.482939    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.501460 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.501419    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-serving-cert\""
Oct 30 18:53:43.502673 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.502650    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:53:43.514479 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514453    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 18:53:43.514628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514482    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:53:43.514628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514486    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 18:53:43.514628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514488    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-graph-raw-hpa-d6eaa-kube-rbac-proxy-sar-config\""
Oct 30 18:53:43.514628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514500    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 18:53:43.514848 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.514565    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 18:53:43.521802 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.521773    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.547887 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.547848    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.548070 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.547905    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkclz\" (UniqueName: \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.548070 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.547960    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.548070 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.548035    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.548174 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.548076    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649364 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649242    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649364 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649314    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkclz\" (UniqueName: \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649364 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649346    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649602 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649372    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649602 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649411    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.649788 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.649769    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.650094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.650071    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.650233 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.650215    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.651964 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.651946    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.671457 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.671417    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkclz\" (UniqueName: \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\") pod \"isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.780106 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.780069    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:43.800328 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.800292    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:43.934563 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:43.934478    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.114352 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:44.114323    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:53:44.117375 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:53:44.117351    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b1de6a_ff4f_4f95_9ca8_48b289b3af6a.slice/crio-93a726ce4dea4df41ea12b57ff18ad7e102d748498437aacb995f64ee159a4fd WatchSource:0}: Error finding container 93a726ce4dea4df41ea12b57ff18ad7e102d748498437aacb995f64ee159a4fd: Status 404 returned error can't find the container with id 93a726ce4dea4df41ea12b57ff18ad7e102d748498437aacb995f64ee159a4fd
Oct 30 18:53:44.395549 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:44.395512    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerStarted","Data":"1c4d044f1065c34beb663cb0854dea89afd2b60515ffc78f5ab3cb0824a8ee74"}
Oct 30 18:53:44.395737 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:44.395567    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerStarted","Data":"93a726ce4dea4df41ea12b57ff18ad7e102d748498437aacb995f64ee159a4fd"}
Oct 30 18:53:48.414100 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:48.414061    4938 generic.go:356] "Generic (PLEG): container finished" podID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerID="1c4d044f1065c34beb663cb0854dea89afd2b60515ffc78f5ab3cb0824a8ee74" exitCode=0
Oct 30 18:53:48.414531 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:48.414135    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerDied","Data":"1c4d044f1065c34beb663cb0854dea89afd2b60515ffc78f5ab3cb0824a8ee74"}
Oct 30 18:53:49.116230 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.116189    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:49.116419 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.116189    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:49.421299 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.421179    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerStarted","Data":"1f95eb20173d20fa4ec8b4ec0ac42113cbdfc567f58f96febdcfbbdb3802946c"}
Oct 30 18:53:49.421299 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.421243    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerStarted","Data":"64b91e6f1ebec6c4922242564125eee81fa003cca2e37df38b8afec6b328967c"}
Oct 30 18:53:49.421726 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.421450    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:49.483130 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:49.483071    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podStartSLOduration=2.625568936 podStartE2EDuration="6.48305405s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.857485114s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:49.482078666 +0000 UTC m=+991.242087127" watchObservedRunningTime="2026-10-30 18:53:49.48305405 +0000 UTC m=+991.243062513"
Oct 30 18:53:50.425740 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:50.425705    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:50.427029 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:50.427002    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:53:51.429841 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:51.429798    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:53:56.435050 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:56.435016    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:53:56.435640 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:56.435608    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:53:59.116182 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:59.116134    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:53:59.116628 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:59.116135    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused"
Oct 30 18:53:59.387758 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:53:59.387666    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:54:02.549596 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:54:02.549555    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:54:06.436454 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:06.436407    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:09.117538 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:09.117507    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:54:09.388480 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:09.388393    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:54:09.952130 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:09.952093    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:54:09.952396 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:09.952366    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container" containerID="cri-o://dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb" gracePeriod=30
Oct 30 18:54:09.952481 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:09.952396    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kube-rbac-proxy" containerID="cri-o://887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269" gracePeriod=30
Oct 30 18:54:10.359664 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:10.359630    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:10.509701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:10.509667    4938 generic.go:356] "Generic (PLEG): container finished" podID="c458501d-080b-40b6-bfbd-b534f55fa813" containerID="887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269" exitCode=0
Oct 30 18:54:10.509880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:10.509709    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerDied","Data":"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"}
Oct 30 18:54:14.109354 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:14.109293    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused"
Oct 30 18:54:15.301442 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.301416    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:54:15.366843 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.366813    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\") pod \"c458501d-080b-40b6-bfbd-b534f55fa813\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") "
Oct 30 18:54:15.367032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.366880    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\") pod \"c458501d-080b-40b6-bfbd-b534f55fa813\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") "
Oct 30 18:54:15.367032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.366910    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\") pod \"c458501d-080b-40b6-bfbd-b534f55fa813\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") "
Oct 30 18:54:15.367032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.366961    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\" (UniqueName: \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\") pod \"c458501d-080b-40b6-bfbd-b534f55fa813\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") "
Oct 30 18:54:15.367032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.367014    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\") pod \"c458501d-080b-40b6-bfbd-b534f55fa813\" (UID: \"c458501d-080b-40b6-bfbd-b534f55fa813\") "
Oct 30 18:54:15.367312 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.367277    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config" pod "c458501d-080b-40b6-bfbd-b534f55fa813" (UID: "c458501d-080b-40b6-bfbd-b534f55fa813"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:15.367454 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.367302    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert" pod "c458501d-080b-40b6-bfbd-b534f55fa813" (UID: "c458501d-080b-40b6-bfbd-b534f55fa813"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:15.367454 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.367443    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location" pod "c458501d-080b-40b6-bfbd-b534f55fa813" (UID: "c458501d-080b-40b6-bfbd-b534f55fa813"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:15.369083 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.369056    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r" pod "c458501d-080b-40b6-bfbd-b534f55fa813" (UID: "c458501d-080b-40b6-bfbd-b534f55fa813"). InnerVolumeSpecName "kube-api-access-nw74r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:15.369083 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.369068    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls" pod "c458501d-080b-40b6-bfbd-b534f55fa813" (UID: "c458501d-080b-40b6-bfbd-b534f55fa813"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:15.468302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.468231    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:15.468302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.468299    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c458501d-080b-40b6-bfbd-b534f55fa813-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:15.468302 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.468314    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw74r\" (UniqueName: \"kubernetes.io/projected/c458501d-080b-40b6-bfbd-b534f55fa813-kube-api-access-nw74r\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:15.468535 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.468328    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c458501d-080b-40b6-bfbd-b534f55fa813-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:15.468535 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.468344    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c458501d-080b-40b6-bfbd-b534f55fa813-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:15.532848 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.532813    4938 generic.go:356] "Generic (PLEG): container finished" podID="c458501d-080b-40b6-bfbd-b534f55fa813" containerID="dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb" exitCode=0
Oct 30 18:54:15.533028 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.532886    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"
Oct 30 18:54:15.533028 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.532889    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerDied","Data":"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"}
Oct 30 18:54:15.533028 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.532951    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g" event={"ID":"c458501d-080b-40b6-bfbd-b534f55fa813","Type":"ContainerDied","Data":"2a8eebec07b0206204a9f0bd057847419c6d1f105c93c959a11dab52924bd051"}
Oct 30 18:54:15.533028 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.532979    4938 scope.go:122] "RemoveContainer" containerID="887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"
Oct 30 18:54:15.542834 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.542812    4938 scope.go:122] "RemoveContainer" containerID="dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"
Oct 30 18:54:15.551172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.551150    4938 scope.go:122] "RemoveContainer" containerID="289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"
Oct 30 18:54:15.559924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.559898    4938 scope.go:122] "RemoveContainer" containerID="887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"
Oct 30 18:54:15.560267 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:54:15.560217    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269\": container with ID starting with 887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269 not found: ID does not exist" containerID="887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"
Oct 30 18:54:15.560393 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.560272    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269"} err="failed to get container status \"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269\": rpc error: code = NotFound desc = could not find container \"887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269\": container with ID starting with 887babb21e81653569b354323b5a051a97ecd421108f0179bf9c2e38ecb2d269 not found: ID does not exist"
Oct 30 18:54:15.560393 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.560293    4938 scope.go:122] "RemoveContainer" containerID="dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"
Oct 30 18:54:15.560791 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:54:15.560768    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb\": container with ID starting with dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb not found: ID does not exist" containerID="dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"
Oct 30 18:54:15.560883 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.560795    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb"} err="failed to get container status \"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb\": rpc error: code = NotFound desc = could not find container \"dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb\": container with ID starting with dcaa5d0f3ef9879a102b32489b72c6e13cc853494d6fb23c7e732585409b8afb not found: ID does not exist"
Oct 30 18:54:15.560883 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.560813    4938 scope.go:122] "RemoveContainer" containerID="289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"
Oct 30 18:54:15.561131 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:54:15.561109    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69\": container with ID starting with 289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69 not found: ID does not exist" containerID="289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"
Oct 30 18:54:15.561190 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.561140    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69"} err="failed to get container status \"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69\": rpc error: code = NotFound desc = could not find container \"289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69\": container with ID starting with 289df5fa9a7d7f0fad852f5306204d3b3470cd6715437adc7095d555cdab4c69 not found: ID does not exist"
Oct 30 18:54:15.587009 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.586974    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:54:15.590096 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.590065    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-predictor-77b4bb9d84-7s98g"]
Oct 30 18:54:15.974406 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.974372    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:54:15.974646 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.974609    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" containerID="cri-o://8523c4edc7c2f12b2fcada92d465096eacdadf9b99f1d858b193bdca6d015c7f" gracePeriod=30
Oct 30 18:54:15.974795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:15.974672    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy" containerID="cri-o://0f2a87cdff6268c43ab7ea70cba023b00f8d92a80a3da9c0aa01c075771c1834" gracePeriod=30
Oct 30 18:54:16.004546 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.004517    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:16.145261 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.145225    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:16.303613 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.303536    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:16.393756 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.393720    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" path="/var/lib/kubelet/pods/c458501d-080b-40b6-bfbd-b534f55fa813/volumes"
Oct 30 18:54:16.435699 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.435660    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:16.540447 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.540414    4938 generic.go:356] "Generic (PLEG): container finished" podID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerID="0f2a87cdff6268c43ab7ea70cba023b00f8d92a80a3da9c0aa01c075771c1834" exitCode=0
Oct 30 18:54:16.540616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:16.540490    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerDied","Data":"0f2a87cdff6268c43ab7ea70cba023b00f8d92a80a3da9c0aa01c075771c1834"}
Oct 30 18:54:19.109910 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:19.109858    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Oct 30 18:54:19.388217 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:19.388132    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused"
Oct 30 18:54:24.110019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.109967    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused"
Oct 30 18:54:24.575438 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.575405    4938 generic.go:356] "Generic (PLEG): container finished" podID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerID="8523c4edc7c2f12b2fcada92d465096eacdadf9b99f1d858b193bdca6d015c7f" exitCode=0
Oct 30 18:54:24.575604 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.575452    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerDied","Data":"8523c4edc7c2f12b2fcada92d465096eacdadf9b99f1d858b193bdca6d015c7f"}
Oct 30 18:54:24.616324 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.616297    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:54:24.638550 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638522    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\") pod \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") "
Oct 30 18:54:24.638718 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638582    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\") pod \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") "
Oct 30 18:54:24.638718 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638623    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\") pod \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") "
Oct 30 18:54:24.638718 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638680    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\" (UniqueName: \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\") pod \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") "
Oct 30 18:54:24.638718 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638704    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\") pod \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\" (UID: \"0abd299c-cd4d-4328-ac8f-f0267848a4c5\") "
Oct 30 18:54:24.638888 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638816    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location" pod "0abd299c-cd4d-4328-ac8f-f0267848a4c5" (UID: "0abd299c-cd4d-4328-ac8f-f0267848a4c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:24.638977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.638960    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:24.639058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.639034    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config" pod "0abd299c-cd4d-4328-ac8f-f0267848a4c5" (UID: "0abd299c-cd4d-4328-ac8f-f0267848a4c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:24.639164 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.639141    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert" pod "0abd299c-cd4d-4328-ac8f-f0267848a4c5" (UID: "0abd299c-cd4d-4328-ac8f-f0267848a4c5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:24.640653 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.640627    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp" pod "0abd299c-cd4d-4328-ac8f-f0267848a4c5" (UID: "0abd299c-cd4d-4328-ac8f-f0267848a4c5"). InnerVolumeSpecName "kube-api-access-z26zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:24.640760 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.640743    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls" pod "0abd299c-cd4d-4328-ac8f-f0267848a4c5" (UID: "0abd299c-cd4d-4328-ac8f-f0267848a4c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:24.739795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.739708    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0abd299c-cd4d-4328-ac8f-f0267848a4c5-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:24.739795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.739740    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:24.739795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.739750    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z26zp\" (UniqueName: \"kubernetes.io/projected/0abd299c-cd4d-4328-ac8f-f0267848a4c5-kube-api-access-z26zp\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:24.739795 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:24.739759    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0abd299c-cd4d-4328-ac8f-f0267848a4c5-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:54:25.581419 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.581388    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"
Oct 30 18:54:25.581878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.581392    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg" event={"ID":"0abd299c-cd4d-4328-ac8f-f0267848a4c5","Type":"ContainerDied","Data":"20881f931100a643cc45855b469c5f2f8ade299bf18793486ba66a50e02e2781"}
Oct 30 18:54:25.581878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.581530    4938 scope.go:122] "RemoveContainer" containerID="0f2a87cdff6268c43ab7ea70cba023b00f8d92a80a3da9c0aa01c075771c1834"
Oct 30 18:54:25.589917 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.589898    4938 scope.go:122] "RemoveContainer" containerID="8523c4edc7c2f12b2fcada92d465096eacdadf9b99f1d858b193bdca6d015c7f"
Oct 30 18:54:25.597342 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.597325    4938 scope.go:122] "RemoveContainer" containerID="93809762dd67d90c55d5653749ab5e32f179bfb308a0475f92d3be0116ea20c2"
Oct 30 18:54:25.649006 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.648970    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:54:25.661873 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:25.661840    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-canary-stop-v2-predictor-784d7544fc-p74zg"]
Oct 30 18:54:26.392674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.392630    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" path="/var/lib/kubelet/pods/0abd299c-cd4d-4328-ac8f-f0267848a4c5/volumes"
Oct 30 18:54:26.436010 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.435963    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:26.732038 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.731955    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:26.820599 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820556    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:54:26.820878 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820865    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="storage-initializer"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820880    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="storage-initializer"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820895    4938 cpu_manager.go:414] "containerMap: removing container" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="storage-initializer"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820901    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="storage-initializer"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820909    4938 cpu_manager.go:414] "containerMap: removing container" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820914    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy"
Oct 30 18:54:26.820924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820926    4938 cpu_manager.go:414] "containerMap: removing container" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820931    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820942    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820947    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820960    4938 cpu_manager.go:414] "containerMap: removing container" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kube-rbac-proxy"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.820965    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kube-rbac-proxy"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.821013    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kube-rbac-proxy"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.821023    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kube-rbac-proxy"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.821029    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="0abd299c-cd4d-4328-ac8f-f0267848a4c5" containerName="kserve-container"
Oct 30 18:54:26.821094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.821037    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="c458501d-080b-40b6-bfbd-b534f55fa813" containerName="kserve-container"
Oct 30 18:54:26.825732 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.825707    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.844721 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.844693    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 18:54:26.856164 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856134    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.856334 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856172    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.856334 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856208    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.856408 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856362    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.856408 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856404    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr4wj\" (UniqueName: \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.856664 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856642    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 18:54:26.856725 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856658    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-primary-ed8d80-kube-rbac-proxy-sar-config\""
Oct 30 18:54:26.856725 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856706    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:54:26.856725 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856717    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"default-dockercfg-gh4qc\""
Oct 30 18:54:26.856856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856759    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-primary-ed8d80-predictor-serving-cert\""
Oct 30 18:54:26.856856 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.856785    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"seaweedfs-s3-creds\""
Oct 30 18:54:26.857808 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.857790    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:54:26.921429 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.921391    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:26.957726 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.957694    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.957924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.957750    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.957924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.957786    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr4wj\" (UniqueName: \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.957924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.957820    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.957924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.957849    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.958200 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.958170    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.958598 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.958578    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.958702 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.958679    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.960330 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.960309    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:26.977839 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:26.977809    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr4wj\" (UniqueName: \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\") pod \"isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") " pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:27.142024 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:27.141989    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:27.453327 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:27.453299    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:54:27.455724 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:54:27.455695    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3275d614_984b_4e17_a711_ce8c0efe5b9f.slice/crio-e281657016183d175b0c626fcac8e1b1637fd6d61bc0bb4dfe8d96fdcee5eb39 WatchSource:0}: Error finding container e281657016183d175b0c626fcac8e1b1637fd6d61bc0bb4dfe8d96fdcee5eb39: Status 404 returned error can't find the container with id e281657016183d175b0c626fcac8e1b1637fd6d61bc0bb4dfe8d96fdcee5eb39
Oct 30 18:54:27.592669 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:27.592633    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerStarted","Data":"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"}
Oct 30 18:54:27.592859 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:27.592703    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerStarted","Data":"e281657016183d175b0c626fcac8e1b1637fd6d61bc0bb4dfe8d96fdcee5eb39"}
Oct 30 18:54:31.611559 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:31.611519    4938 generic.go:356] "Generic (PLEG): container finished" podID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerID="dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92" exitCode=0
Oct 30 18:54:31.612081 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:31.611589    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerDied","Data":"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"}
Oct 30 18:54:32.552951 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:54:32.552912    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:54:32.617815 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:32.617776    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerStarted","Data":"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"}
Oct 30 18:54:32.618205 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:32.617840    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerStarted","Data":"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"}
Oct 30 18:54:32.618205 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:32.618025    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:32.675780 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:32.675718    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podStartSLOduration=2.969908163 podStartE2EDuration="6.675706901s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.705798738s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:32.674852353 +0000 UTC m=+1034.434860818" watchObservedRunningTime="2026-10-30 18:54:32.675706901 +0000 UTC m=+1034.435715361"
Oct 30 18:54:33.621673 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:33.621640    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:33.623052 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:33.623029    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:54:34.626483 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:34.626447    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:54:36.436145 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:36.436106    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:39.631481 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:39.631451    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:54:39.631927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:39.631901    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:54:46.436074 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:46.436024    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:49.632872 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:49.632829    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:54:56.435689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:56.435642    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:54:59.632796 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:54:59.632755    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:55:00.387858 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:00.387803    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.33:8080: connect: connection refused"
Oct 30 18:55:02.554433 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:55:02.554368    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:55:09.632705 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:09.632661    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:55:10.393383 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:10.393356    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:55:10.461804 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:10.461747    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:19.632343 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:19.632303    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:55:29.632877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:29.632835    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:55:32.550216 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:55:32.550174    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:55:33.591267 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:33.591219    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:33.826283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:33.826214    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:39.632830 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:39.632786    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.34:8080: connect: connection refused"
Oct 30 18:55:44.108712 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.108677    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:55:44.109160 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.109040    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container" containerID="cri-o://64b91e6f1ebec6c4922242564125eee81fa003cca2e37df38b8afec6b328967c" gracePeriod=30
Oct 30 18:55:44.109160 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.109076    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kube-rbac-proxy" containerID="cri-o://1f95eb20173d20fa4ec8b4ec0ac42113cbdfc567f58f96febdcfbbdb3802946c" gracePeriod=30
Oct 30 18:55:44.180263 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.180218    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:44.324982 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.324945    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:44.908348 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.908315    4938 generic.go:356] "Generic (PLEG): container finished" podID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerID="1f95eb20173d20fa4ec8b4ec0ac42113cbdfc567f58f96febdcfbbdb3802946c" exitCode=0
Oct 30 18:55:44.908528 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:44.908369    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerDied","Data":"1f95eb20173d20fa4ec8b4ec0ac42113cbdfc567f58f96febdcfbbdb3802946c"}
Oct 30 18:55:46.431024 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:46.430971    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.33:8643/healthz\": dial tcp 10.133.0.33:8643: connect: connection refused"
Oct 30 18:55:48.927098 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:48.927072    4938 generic.go:356] "Generic (PLEG): container finished" podID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerID="64b91e6f1ebec6c4922242564125eee81fa003cca2e37df38b8afec6b328967c" exitCode=0
Oct 30 18:55:48.927379 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:48.927148    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerDied","Data":"64b91e6f1ebec6c4922242564125eee81fa003cca2e37df38b8afec6b328967c"}
Oct 30 18:55:49.053097 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.053068    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:55:49.118614 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118523    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\" (UniqueName: \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\") pod \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") "
Oct 30 18:55:49.118614 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118577    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\") pod \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") "
Oct 30 18:55:49.118614 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118609    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\") pod \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") "
Oct 30 18:55:49.118898 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118704    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\") pod \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") "
Oct 30 18:55:49.118898 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118804    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\") pod \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\" (UID: \"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a\") "
Oct 30 18:55:49.119015 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118930    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location" pod "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" (UID: "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:55:49.119015 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118956    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert" pod "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" (UID: "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:49.119015 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.118998    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config" pod "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" (UID: "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:55:49.119172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.119068    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:55:49.119172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.119080    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:55:49.119172 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.119090    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:55:49.120761 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.120735    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz" pod "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" (UID: "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a"). InnerVolumeSpecName "kube-api-access-pkclz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:55:49.120877 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.120813    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls" pod "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" (UID: "87b1de6a-ff4f-4f95-9ca8-48b289b3af6a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:55:49.220440 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.220401    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:55:49.220440 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.220435    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkclz\" (UniqueName: \"kubernetes.io/projected/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a-kube-api-access-pkclz\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:55:49.632975 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.632945    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:55:49.933479 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.933404    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"
Oct 30 18:55:49.933894 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.933404    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq" event={"ID":"87b1de6a-ff4f-4f95-9ca8-48b289b3af6a","Type":"ContainerDied","Data":"93a726ce4dea4df41ea12b57ff18ad7e102d748498437aacb995f64ee159a4fd"}
Oct 30 18:55:49.933894 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.933540    4938 scope.go:122] "RemoveContainer" containerID="1f95eb20173d20fa4ec8b4ec0ac42113cbdfc567f58f96febdcfbbdb3802946c"
Oct 30 18:55:49.942663 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.942642    4938 scope.go:122] "RemoveContainer" containerID="64b91e6f1ebec6c4922242564125eee81fa003cca2e37df38b8afec6b328967c"
Oct 30 18:55:49.950579 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.950552    4938 scope.go:122] "RemoveContainer" containerID="1c4d044f1065c34beb663cb0854dea89afd2b60515ffc78f5ab3cb0824a8ee74"
Oct 30 18:55:49.988393 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:49.988357    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:55:50.001564 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:50.001528    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-graph-raw-hpa-d6eaa-predictor-685bb5c546-489zq"]
Oct 30 18:55:50.392208 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:50.392174    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" path="/var/lib/kubelet/pods/87b1de6a-ff4f-4f95-9ca8-48b289b3af6a/volumes"
Oct 30 18:55:51.548732 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.548693    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:51.566676 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.566643    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:55:51.567032 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567015    4938 cpu_manager.go:414] "containerMap: removing container" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container"
Oct 30 18:55:51.567117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567035    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container"
Oct 30 18:55:51.567117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567064    4938 cpu_manager.go:414] "containerMap: removing container" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kube-rbac-proxy"
Oct 30 18:55:51.567117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567072    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kube-rbac-proxy"
Oct 30 18:55:51.567117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567095    4938 cpu_manager.go:414] "containerMap: removing container" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="storage-initializer"
Oct 30 18:55:51.567117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567103    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="storage-initializer"
Oct 30 18:55:51.567402 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567199    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kserve-container"
Oct 30 18:55:51.567402 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.567214    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="87b1de6a-ff4f-4f95-9ca8-48b289b3af6a" containerName="kube-rbac-proxy"
Oct 30 18:55:51.571850 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.571826    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.592346 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.592317    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:55:51.604286 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.604256    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"message-dumper-raw-6a446-kube-rbac-proxy-sar-config\""
Oct 30 18:55:51.604464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.604240    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"message-dumper-raw-6a446-predictor-serving-cert\""
Oct 30 18:55:51.604464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.604305    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 18:55:51.604464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.604297    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 18:55:51.604464 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.604333    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 18:55:51.641134 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.641096    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.641376 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.641155    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.641376 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.641298    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vknm\" (UniqueName: \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.694420 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.694387    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:51.742465 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.742430    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.742616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.742485    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vknm\" (UniqueName: \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.742616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.742525    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.743173 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.743153    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.745272 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.745233    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.762474 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.762436    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vknm\" (UniqueName: \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\") pod \"message-dumper-raw-6a446-predictor-6976756b99-mpl92\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") " pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:51.886122 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:51.886082    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:52.184917 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:52.184883    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:55:52.188201 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:55:52.188170    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a7d2fb_7380_432c_a4eb_2777f35150cb.slice/crio-38f671a85ff442f8327aa31dfab18e7420610c3eff84d436f5e418085b64bfdf WatchSource:0}: Error finding container 38f671a85ff442f8327aa31dfab18e7420610c3eff84d436f5e418085b64bfdf: Status 404 returned error can't find the container with id 38f671a85ff442f8327aa31dfab18e7420610c3eff84d436f5e418085b64bfdf
Oct 30 18:55:52.948653 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:52.948619    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerStarted","Data":"38f671a85ff442f8327aa31dfab18e7420610c3eff84d436f5e418085b64bfdf"}
Oct 30 18:55:53.954895 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:53.954849    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerStarted","Data":"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"}
Oct 30 18:55:53.955435 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:53.954922    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerStarted","Data":"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"}
Oct 30 18:55:53.955435 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:53.954982    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:53.998163 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:53.998107    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" podStartSLOduration=1.406351881 podStartE2EDuration="2.99809245s" totalImagesPullingTime="1.591740569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:53.997729158 +0000 UTC m=+1115.757737621" watchObservedRunningTime="2026-10-30 18:55:53.99809245 +0000 UTC m=+1115.758100912"
Oct 30 18:55:54.959651 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:54.959622    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:55:54.961596 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:55:54.961575    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:56:01.973406 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:01.973323    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:56:02.549376 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:02.549338    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:56:10.605412 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:10.605380    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:11.760053 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.760012    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:11.909168 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.909131    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:11.933136 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.933102    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:56:11.937166 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.937145    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:11.958162 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.958131    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 18:56:11.958359 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.958192    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:56:11.958359 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.958193    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-logger-raw-6a446-kube-rbac-proxy-sar-config\""
Oct 30 18:56:11.958359 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.958280    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-logger-raw-6a446-predictor-serving-cert\""
Oct 30 18:56:11.973105 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:11.973073    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:56:12.105038 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.104994    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.105203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.105066    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5skh\" (UniqueName: \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.105203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.105139    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.105203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.105162    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.105203 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.105194    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.206679 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.206634    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.206882 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.206695    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5skh\" (UniqueName: \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.206882 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.206726    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.206882 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.206743    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.206882 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.206772    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.207144 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.207114    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.207526 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.207497    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.207526 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.207514    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.209180 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.209160    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.228030 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.228000    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5skh\" (UniqueName: \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\") pod \"isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") " pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.256918 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.256871    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:12.465997 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.465919    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:12.568849 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.568812    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:12.572597 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.572580    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.586280 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.586240    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:56:12.588324 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.588304    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-secondary-ed8d80-kube-rbac-proxy-sar-config\""
Oct 30 18:56:12.588931 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:56:12.588909    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d423c7_4fa3_4f9d_8df5_763eba9b8643.slice/crio-6d3b86cd69ff9964c1858070ad1027408d1c590498932977bc7248c77f911079 WatchSource:0}: Error finding container 6d3b86cd69ff9964c1858070ad1027408d1c590498932977bc7248c77f911079: Status 404 returned error can't find the container with id 6d3b86cd69ff9964c1858070ad1027408d1c590498932977bc7248c77f911079
Oct 30 18:56:12.598350 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.598325    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"invalid-s3-secret-ed8d80\""
Oct 30 18:56:12.598513 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.598386    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-secondary-ed8d80-predictor-serving-cert\""
Oct 30 18:56:12.598513 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.598449    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"invalid-s3-sa-ed8d80-dockercfg-nd7b4\""
Oct 30 18:56:12.599623 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.599601    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:12.614890 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.614856    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:12.711274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.711217    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.711274 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.711275    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4wq\" (UniqueName: \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.711548 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.711358    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.711548 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.711432    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.711548 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.711491    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812096    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812150    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812728 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812219    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812728 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812267    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4wq\" (UniqueName: \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812728 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812294    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.812728 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812614    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.813008 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812984    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.813057 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.812992    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.814891 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.814868    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.834051 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.834014    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4wq\" (UniqueName: \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\") pod \"isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") " pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:12.891039 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:12.890991    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:13.035001 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:13.034950    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerStarted","Data":"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"}
Oct 30 18:56:13.035205 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:13.035066    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerStarted","Data":"6d3b86cd69ff9964c1858070ad1027408d1c590498932977bc7248c77f911079"}
Oct 30 18:56:13.202056 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:13.202017    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:13.205054 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:56:13.205025    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31acd0c1_ee4d_4ff0_af1f_8d8baf8ee278.slice/crio-dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3 WatchSource:0}: Error finding container dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3: Status 404 returned error can't find the container with id dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3
Oct 30 18:56:14.041283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:14.041223    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerStarted","Data":"616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"}
Oct 30 18:56:14.041792 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:14.041315    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerStarted","Data":"dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3"}
Oct 30 18:56:17.057114 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:17.057075    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerID="26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355" exitCode=0
Oct 30 18:56:17.057563 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:17.057129    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerDied","Data":"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"}
Oct 30 18:56:18.064078 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.063988    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerStarted","Data":"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"}
Oct 30 18:56:18.064078 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.064066    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerStarted","Data":"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"}
Oct 30 18:56:18.064578 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.064105    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerStarted","Data":"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"}
Oct 30 18:56:18.064578 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.064426    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:18.064578 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.064515    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:18.065821 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.065797    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/0.log"
Oct 30 18:56:18.065954 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.065829    4938 generic.go:356] "Generic (PLEG): container finished" podID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerID="616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f" exitCode=1
Oct 30 18:56:18.065954 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.065880    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerDied","Data":"616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"}
Oct 30 18:56:18.065954 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.065912    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:18.136876 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:18.136822    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podStartSLOduration=3.479055567 podStartE2EDuration="7.136806883s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.657751316s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:18.122727946 +0000 UTC m=+1139.882736410" watchObservedRunningTime="2026-10-30 18:56:18.136806883 +0000 UTC m=+1139.896815337"
Oct 30 18:56:19.077862 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:19.077828    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/0.log"
Oct 30 18:56:19.078326 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:19.077977    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerStarted","Data":"4c4322abb7e2e90329244432fa8390ca9b7f3fbf398469dd3bb33059c8abc635"}
Oct 30 18:56:19.078442 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:19.078409    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:19.078522 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:19.078441    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:19.079785 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:19.079754    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:20.082669 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:20.082608    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:20.083143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:20.083049    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:25.086616 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.086585    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:56:25.087301 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.087273    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:25.087515 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.087492    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:25.103675 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.103652    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/1.log"
Oct 30 18:56:25.104094 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.104077    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/0.log"
Oct 30 18:56:25.104161 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.104110    4938 generic.go:356] "Generic (PLEG): container finished" podID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerID="4c4322abb7e2e90329244432fa8390ca9b7f3fbf398469dd3bb33059c8abc635" exitCode=1
Oct 30 18:56:25.104202 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.104184    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerDied","Data":"4c4322abb7e2e90329244432fa8390ca9b7f3fbf398469dd3bb33059c8abc635"}
Oct 30 18:56:25.104279 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.104267    4938 scope.go:122] "RemoveContainer" containerID="616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"
Oct 30 18:56:25.104635 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.104620    4938 scope.go:122] "RemoveContainer" containerID="616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"
Oct 30 18:56:25.115473 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:25.115446    4938 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_e2e-gw1-d1ae4027ce(31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278)\"" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"
Oct 30 18:56:25.115764 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:25.115732    4938 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_e2e-gw1-d1ae4027ce_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278_0 in pod sandbox dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3 from index: no such id: '616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f'" containerID="616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"
Oct 30 18:56:25.115861 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:25.115772    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_e2e-gw1-d1ae4027ce_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278_0 in pod sandbox dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3 from index: no such id: '616495f39080fc8fa69ac2609b29f4786f1e0e53021a5ce642bc72d4d4ef101f'"
Oct 30 18:56:26.110113 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:26.110083    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/1.log"
Oct 30 18:56:30.451875 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.451841    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:30.528623 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.528535    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.551283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.551219    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:56:30.551552 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.551520    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container" containerID="cri-o://ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9" gracePeriod=30
Oct 30 18:56:30.551552 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.551542    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kube-rbac-proxy" containerID="cri-o://30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25" gracePeriod=30
Oct 30 18:56:30.672966 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.672943    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/1.log"
Oct 30 18:56:30.675495 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.675479    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:30.676982 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.676958    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:30.762281 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762177    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\" (UniqueName: \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\") pod \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") "
Oct 30 18:56:30.762281 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762225    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\") pod \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") "
Oct 30 18:56:30.762522 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762289    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\" (UniqueName: \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\") pod \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") "
Oct 30 18:56:30.762522 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762326    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\") pod \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") "
Oct 30 18:56:30.762522 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762377    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\") pod \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\" (UID: \"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278\") "
Oct 30 18:56:30.762697 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762655    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert" pod "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" (UID: "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:30.762779 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762700    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config" pod "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" (UID: "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:30.762837 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.762781    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location" pod "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" (UID: "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:30.764487 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.764460    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq" pod "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" (UID: "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"). InnerVolumeSpecName "kube-api-access-6p4wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:30.764607 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.764513    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls" pod "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" (UID: "31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:30.863188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.863146    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:30.863188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.863177    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:30.863188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.863187    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p4wq\" (UniqueName: \"kubernetes.io/projected/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-api-access-6p4wq\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:30.863188 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.863198    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:30.863566 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:30.863207    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:31.131233 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.131194    4938 generic.go:356] "Generic (PLEG): container finished" podID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerID="30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25" exitCode=0
Oct 30 18:56:31.131429 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.131278    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerDied","Data":"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"}
Oct 30 18:56:31.132654 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.132637    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv_31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/storage-initializer/1.log"
Oct 30 18:56:31.132751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.132735    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"
Oct 30 18:56:31.132751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.132736    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv" event={"ID":"31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278","Type":"ContainerDied","Data":"dd585dba76fc3c325a6d406b9dea801dd13ef391d9f1a2773deba26c0af954f3"}
Oct 30 18:56:31.132826 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.132801    4938 scope.go:122] "RemoveContainer" containerID="4c4322abb7e2e90329244432fa8390ca9b7f3fbf398469dd3bb33059c8abc635"
Oct 30 18:56:31.217864 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.217824    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:31.221823 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:31.221792    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-secondary-ed8d80-predictor-7f584c575f-z2hpv"]
Oct 30 18:56:32.393590 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:32.393556    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" path="/var/lib/kubelet/pods/31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278/volumes"
Oct 30 18:56:32.550291 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:32.550241    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:56:34.627132 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:34.627093    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.34:8643/healthz\": dial tcp 10.133.0.34:8643: connect: connection refused"
Oct 30 18:56:35.087255 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.087194    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:35.087723 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.087697    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:35.809372 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.809343    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:56:35.894760 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.894723    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\") pod \"3275d614-984b-4e17-a711-ce8c0efe5b9f\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") "
Oct 30 18:56:35.894760 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.894761    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\" (UniqueName: \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\") pod \"3275d614-984b-4e17-a711-ce8c0efe5b9f\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") "
Oct 30 18:56:35.895009 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.894815    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\") pod \"3275d614-984b-4e17-a711-ce8c0efe5b9f\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") "
Oct 30 18:56:35.895009 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.894835    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\") pod \"3275d614-984b-4e17-a711-ce8c0efe5b9f\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") "
Oct 30 18:56:35.895009 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.894857    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\" (UniqueName: \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\") pod \"3275d614-984b-4e17-a711-ce8c0efe5b9f\" (UID: \"3275d614-984b-4e17-a711-ce8c0efe5b9f\") "
Oct 30 18:56:35.895208 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.895181    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location" pod "3275d614-984b-4e17-a711-ce8c0efe5b9f" (UID: "3275d614-984b-4e17-a711-ce8c0efe5b9f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:35.895303 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.895267    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config" pod "3275d614-984b-4e17-a711-ce8c0efe5b9f" (UID: "3275d614-984b-4e17-a711-ce8c0efe5b9f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:35.895346 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.895276    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert" pod "3275d614-984b-4e17-a711-ce8c0efe5b9f" (UID: "3275d614-984b-4e17-a711-ce8c0efe5b9f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:35.897012 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.896992    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls" pod "3275d614-984b-4e17-a711-ce8c0efe5b9f" (UID: "3275d614-984b-4e17-a711-ce8c0efe5b9f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:35.897105 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.897014    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj" pod "3275d614-984b-4e17-a711-ce8c0efe5b9f" (UID: "3275d614-984b-4e17-a711-ce8c0efe5b9f"). InnerVolumeSpecName "kube-api-access-sr4wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:35.995627 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.995532    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3275d614-984b-4e17-a711-ce8c0efe5b9f-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:35.995627 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.995565    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3275d614-984b-4e17-a711-ce8c0efe5b9f-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:35.995627 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.995577    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:35.995627 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.995587    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3275d614-984b-4e17-a711-ce8c0efe5b9f-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:35.995627 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:35.995595    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr4wj\" (UniqueName: \"kubernetes.io/projected/3275d614-984b-4e17-a711-ce8c0efe5b9f-kube-api-access-sr4wj\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:36.157815 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.157780    4938 generic.go:356] "Generic (PLEG): container finished" podID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerID="ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9" exitCode=0
Oct 30 18:56:36.157977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.157832    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerDied","Data":"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"}
Oct 30 18:56:36.157977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.157839    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"
Oct 30 18:56:36.157977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.157880    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb" event={"ID":"3275d614-984b-4e17-a711-ce8c0efe5b9f","Type":"ContainerDied","Data":"e281657016183d175b0c626fcac8e1b1637fd6d61bc0bb4dfe8d96fdcee5eb39"}
Oct 30 18:56:36.157977 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.157909    4938 scope.go:122] "RemoveContainer" containerID="30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"
Oct 30 18:56:36.166689 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.166667    4938 scope.go:122] "RemoveContainer" containerID="ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"
Oct 30 18:56:36.174787 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.174767    4938 scope.go:122] "RemoveContainer" containerID="dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"
Oct 30 18:56:36.182117 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182099    4938 scope.go:122] "RemoveContainer" containerID="30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"
Oct 30 18:56:36.182413 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:36.182391    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25\": container with ID starting with 30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25 not found: ID does not exist" containerID="30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"
Oct 30 18:56:36.182497 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182429    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25"} err="failed to get container status \"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25\": rpc error: code = NotFound desc = could not find container \"30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25\": container with ID starting with 30af45b21f6d2be7a2ddf16eeb3496adcd33d7faf1ddb8bdb4c0c5e16bb99e25 not found: ID does not exist"
Oct 30 18:56:36.182497 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182458    4938 scope.go:122] "RemoveContainer" containerID="ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"
Oct 30 18:56:36.182711 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:36.182692    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9\": container with ID starting with ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9 not found: ID does not exist" containerID="ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"
Oct 30 18:56:36.182757 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182718    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9"} err="failed to get container status \"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9\": rpc error: code = NotFound desc = could not find container \"ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9\": container with ID starting with ee2ae3f3844641de13d7b7b195cb97da96f8f593e917295aef4dab463efffdd9 not found: ID does not exist"
Oct 30 18:56:36.182757 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182735    4938 scope.go:122] "RemoveContainer" containerID="dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"
Oct 30 18:56:36.182944 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:36.182926    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92\": container with ID starting with dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92 not found: ID does not exist" containerID="dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"
Oct 30 18:56:36.182998 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.182954    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92"} err="failed to get container status \"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92\": rpc error: code = NotFound desc = could not find container \"dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92\": container with ID starting with dff421942d59fba598707bcc2e010a8dbe3469fed849a8e5b55257105c1a7f92 not found: ID does not exist"
Oct 30 18:56:36.212800 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.212770    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:56:36.216417 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.216395    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-primary-ed8d80-predictor-554fd8fc68-9kgjb"]
Oct 30 18:56:36.398165 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:36.398126    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" path="/var/lib/kubelet/pods/3275d614-984b-4e17-a711-ce8c0efe5b9f/volumes"
Oct 30 18:56:38.202983 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.202947    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.295723 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.295688    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:38.296285 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296237    4938 cpu_manager.go:414] "containerMap: removing container" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296285 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296286    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296298    4938 cpu_manager.go:414] "containerMap: removing container" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296304    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296338    4938 cpu_manager.go:414] "containerMap: removing container" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="storage-initializer"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296344    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="storage-initializer"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296351    4938 cpu_manager.go:414] "containerMap: removing container" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296356    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296361    4938 cpu_manager.go:414] "containerMap: removing container" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kube-rbac-proxy"
Oct 30 18:56:38.296413 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296365    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kube-rbac-proxy"
Oct 30 18:56:38.296649 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296422    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296649 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296429    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="31acd0c1-ee4d-4ff0-af1f-8d8baf8ee278" containerName="storage-initializer"
Oct 30 18:56:38.296649 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296441    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kserve-container"
Oct 30 18:56:38.296649 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.296447    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="3275d614-984b-4e17-a711-ce8c0efe5b9f" containerName="kube-rbac-proxy"
Oct 30 18:56:38.299643 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.299625    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.312352 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.312324    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2fxh\" (UniqueName: \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.312459 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.312366    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.312459 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.312386    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.312536 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.312479    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.312536 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.312506    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.328769 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328739    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"fail-s3-secret-36e552\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328814    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-init-fail-36e552-predictor-serving-cert\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328815    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"fail-s3-sa-36e552-dockercfg-mlst5\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328815    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328845    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"isvc-init-fail-36e552-kube-rbac-proxy-sar-config\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328879    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 18:56:38.328924 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.328911    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 18:56:38.330528 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.330510    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:38.346818 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.346783    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.413020 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.412987    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413198 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413028    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413260 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413209    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2fxh\" (UniqueName: \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413323 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413303    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413386 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413338    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413476 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413453    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413790 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413767    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.413901 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.413886    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.415633 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.415614    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.433566 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.433533    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2fxh\" (UniqueName: \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\") pod \"isvc-init-fail-36e552-predictor-6997c95977-chblb\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") " pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.616261 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.616207    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:38.928786 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:38.928758    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:38.931048 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:56:38.931013    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac542c6f_ac57_4b28_87b0_8962d07f162b.slice/crio-4dd678cade6995d8ad98fa9a81c1c8c66826340624fe793bd65ea5529c30959d WatchSource:0}: Error finding container 4dd678cade6995d8ad98fa9a81c1c8c66826340624fe793bd65ea5529c30959d: Status 404 returned error can't find the container with id 4dd678cade6995d8ad98fa9a81c1c8c66826340624fe793bd65ea5529c30959d
Oct 30 18:56:39.174284 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:39.174231    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerStarted","Data":"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"}
Oct 30 18:56:39.174456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:39.174310    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerStarted","Data":"4dd678cade6995d8ad98fa9a81c1c8c66826340624fe793bd65ea5529c30959d"}
Oct 30 18:56:45.087695 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:45.087645    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:45.088204 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:45.088180    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:45.200569 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:45.200542    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/0.log"
Oct 30 18:56:45.200732 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:45.200579    4938 generic.go:356] "Generic (PLEG): container finished" podID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerID="7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b" exitCode=1
Oct 30 18:56:45.200732 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:45.200664    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerDied","Data":"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"}
Oct 30 18:56:46.207283 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:46.207239    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/0.log"
Oct 30 18:56:46.207704 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:46.207338    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerStarted","Data":"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"}
Oct 30 18:56:48.273301 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:48.273264    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:48.273703 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:48.273545    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer" containerID="cri-o://903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6" gracePeriod=30
Oct 30 18:56:48.348602 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:48.348560    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:50.623697 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.623671    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/1.log"
Oct 30 18:56:50.624073 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.624058    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/0.log"
Oct 30 18:56:50.626351 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.626336    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:50.713956 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.713866    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\") pod \"ac542c6f-ac57-4b28-87b0-8962d07f162b\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") "
Oct 30 18:56:50.713956 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.713916    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\") pod \"ac542c6f-ac57-4b28-87b0-8962d07f162b\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") "
Oct 30 18:56:50.713956 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.713944    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\" (UniqueName: \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\") pod \"ac542c6f-ac57-4b28-87b0-8962d07f162b\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") "
Oct 30 18:56:50.714219 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.713976    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\") pod \"ac542c6f-ac57-4b28-87b0-8962d07f162b\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") "
Oct 30 18:56:50.714219 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.714024    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\") pod \"ac542c6f-ac57-4b28-87b0-8962d07f162b\" (UID: \"ac542c6f-ac57-4b28-87b0-8962d07f162b\") "
Oct 30 18:56:50.714369 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.714306    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location" pod "ac542c6f-ac57-4b28-87b0-8962d07f162b" (UID: "ac542c6f-ac57-4b28-87b0-8962d07f162b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:56:50.714435 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.714397    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert" pod "ac542c6f-ac57-4b28-87b0-8962d07f162b" (UID: "ac542c6f-ac57-4b28-87b0-8962d07f162b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:50.714525 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.714500    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config" pod "ac542c6f-ac57-4b28-87b0-8962d07f162b" (UID: "ac542c6f-ac57-4b28-87b0-8962d07f162b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:56:50.716097 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.716076    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls" pod "ac542c6f-ac57-4b28-87b0-8962d07f162b" (UID: "ac542c6f-ac57-4b28-87b0-8962d07f162b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:56:50.716204 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.716189    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh" pod "ac542c6f-ac57-4b28-87b0-8962d07f162b" (UID: "ac542c6f-ac57-4b28-87b0-8962d07f162b"). InnerVolumeSpecName "kube-api-access-b2fxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:56:50.814889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.814848    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ac542c6f-ac57-4b28-87b0-8962d07f162b-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:50.814889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.814880    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:50.814889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.814890    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac542c6f-ac57-4b28-87b0-8962d07f162b-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:50.814889 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.814899    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ac542c6f-ac57-4b28-87b0-8962d07f162b-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:50.815205 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:50.814907    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2fxh\" (UniqueName: \"kubernetes.io/projected/ac542c6f-ac57-4b28-87b0-8962d07f162b-kube-api-access-b2fxh\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:56:51.228568 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.228532    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/1.log"
Oct 30 18:56:51.228938 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.228922    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-d1ae4027ce_isvc-init-fail-36e552-predictor-6997c95977-chblb_ac542c6f-ac57-4b28-87b0-8962d07f162b/storage-initializer/0.log"
Oct 30 18:56:51.228987 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.228956    4938 generic.go:356] "Generic (PLEG): container finished" podID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerID="903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6" exitCode=1
Oct 30 18:56:51.229022 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.228983    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerDied","Data":"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"}
Oct 30 18:56:51.229059 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.229026    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"
Oct 30 18:56:51.229059 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.229039    4938 scope.go:122] "RemoveContainer" containerID="903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"
Oct 30 18:56:51.229156 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.229030    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb" event={"ID":"ac542c6f-ac57-4b28-87b0-8962d07f162b","Type":"ContainerDied","Data":"4dd678cade6995d8ad98fa9a81c1c8c66826340624fe793bd65ea5529c30959d"}
Oct 30 18:56:51.237472 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.237450    4938 scope.go:122] "RemoveContainer" containerID="7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"
Oct 30 18:56:51.245289 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.245265    4938 scope.go:122] "RemoveContainer" containerID="903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"
Oct 30 18:56:51.245578 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:51.245555    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6\": container with ID starting with 903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6 not found: ID does not exist" containerID="903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"
Oct 30 18:56:51.245626 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.245592    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6"} err="failed to get container status \"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6\": rpc error: code = NotFound desc = could not find container \"903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6\": container with ID starting with 903eccbd0eee58c5bb0f2dd4faff474eca6af4b3f28dcc110b3534213c7b0af6 not found: ID does not exist"
Oct 30 18:56:51.245626 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.245619    4938 scope.go:122] "RemoveContainer" containerID="7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"
Oct 30 18:56:51.245903 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:56:51.245884    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b\": container with ID starting with 7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b not found: ID does not exist" containerID="7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"
Oct 30 18:56:51.245962 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.245908    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b"} err="failed to get container status \"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b\": rpc error: code = NotFound desc = could not find container \"7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b\": container with ID starting with 7bb6eaf0c8cceaaf89bc44d9421c333d84cada11bfc164e10aa5c138f5e62e8b not found: ID does not exist"
Oct 30 18:56:51.327239 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.327199    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:51.332670 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:51.332642    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/isvc-init-fail-36e552-predictor-6997c95977-chblb"]
Oct 30 18:56:52.392213 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:52.392179    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" path="/var/lib/kubelet/pods/ac542c6f-ac57-4b28-87b0-8962d07f162b/volumes"
Oct 30 18:56:53.932433 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:53.932388    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:54.023577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.023541    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:56:54.023929 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.023916    4938 cpu_manager.go:414] "containerMap: removing container" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.023981 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.023931    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.024015 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.024010    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.024045 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.024023    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.024102 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.024092    4938 cpu_manager.go:414] "containerMap: removing container" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.024137 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.024102    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac542c6f-ac57-4b28-87b0-8962d07f162b" containerName="storage-initializer"
Oct 30 18:56:54.028847 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.028825    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.045660 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.045627    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"raw-sklearn-a3b71-kube-rbac-proxy-sar-config\""
Oct 30 18:56:54.045660 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.045646    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:56:54.045660 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.045627    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 18:56:54.058746 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.058699    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 18:56:54.058926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.058791    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"seaweedfs-s3-creds\""
Oct 30 18:56:54.058926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.058804    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"default-dockercfg-gh4qc\""
Oct 30 18:56:54.058926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.058805    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"raw-sklearn-a3b71-predictor-serving-cert\""
Oct 30 18:56:54.060106 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.060072    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:56:54.075894 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.075854    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:54.142145 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.142105    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.142240 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.142153    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.142240 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.142173    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.142349 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.142295    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.142385 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.142359    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhpdw\" (UniqueName: \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243052    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243095    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243143 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243127    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243405 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243189    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhpdw\" (UniqueName: \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243405 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243209    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243523 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243503    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243841 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243816    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.243946 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.243907    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.245674 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.245651    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.263577 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.263550    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhpdw\" (UniqueName: \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\") pod \"raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.345683 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.345645    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:56:54.657857 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:54.657831    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:56:54.661240 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:56:54.661201    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56899e54_54ed_407f_8adf_2e07fd2355de.slice/crio-f343b7587ad2b3813584f2cb467107f0e34536b8c6508b06fc41050137729cd7 WatchSource:0}: Error finding container f343b7587ad2b3813584f2cb467107f0e34536b8c6508b06fc41050137729cd7: Status 404 returned error can't find the container with id f343b7587ad2b3813584f2cb467107f0e34536b8c6508b06fc41050137729cd7
Oct 30 18:56:55.030079 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:55.029989    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:55.087519 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:55.087464    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:56:55.087926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:55.087897    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:56:55.248853 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:55.248809    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerStarted","Data":"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"}
Oct 30 18:56:55.249056 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:55.248888    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerStarted","Data":"f343b7587ad2b3813584f2cb467107f0e34536b8c6508b06fc41050137729cd7"}
Oct 30 18:56:59.265846 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:59.265813    4938 generic.go:356] "Generic (PLEG): container finished" podID="56899e54-54ed-407f-8adf-2e07fd2355de" containerID="331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af" exitCode=0
Oct 30 18:56:59.266291 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:56:59.265885    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerDied","Data":"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"}
Oct 30 18:57:00.271849 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:00.271807    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerStarted","Data":"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"}
Oct 30 18:57:00.272275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:00.271897    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerStarted","Data":"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"}
Oct 30 18:57:00.272275 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:00.272113    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:57:00.333178 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:00.333125    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podStartSLOduration=3.447685844 podStartE2EDuration="7.33311214s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.885426296s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:00.330659848 +0000 UTC m=+1182.090668312" watchObservedRunningTime="2026-10-30 18:57:00.33311214 +0000 UTC m=+1182.093120597"
Oct 30 18:57:01.276080 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:01.276036    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:57:01.277398 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:01.277372    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:02.279431 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:02.279390    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:02.549626 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:57:02.549535    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:57:05.087701 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:05.087654    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:57:05.088189 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:05.088160    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:07.284517 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:07.284483    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:57:07.285002 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:07.284974    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:10.714921 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:10.714879    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:15.088048 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:15.088002    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:57:15.088599 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:15.088484    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:17.285929 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:17.285888    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:18.497163 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:18.497136    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:57:18.499386 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:18.499363    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 18:57:18.501405 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:18.501382    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:57:18.503625 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:18.503604    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 18:57:25.087264 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:25.087214    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:57:25.087751 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:25.087669    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:27.285321 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:27.285277    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:32.550782 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:57:32.550700    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:57:35.087371 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:35.087313    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:57:35.087857 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:35.087784    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:57:37.285715 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:37.285673    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:45.088422 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:45.088381    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:57:45.088863 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:45.088500    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:57:47.285150 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:47.285109    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:57.065425 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.065389    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-42351c4657_message-dumper-raw-6a446-predictor-6976756b99-mpl92_11a7d2fb-7380-432c-a4eb-2777f35150cb/kserve-container/0.log"
Oct 30 18:57:57.285016 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.284972    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:57:57.640588 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.640551    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:57:57.640926 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.640890    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" containerID="cri-o://ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975" gracePeriod=30
Oct 30 18:57:57.641199 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.640890    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" containerID="cri-o://4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf" gracePeriod=30
Oct 30 18:57:57.641329 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.640937    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" containerID="cri-o://59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e" gracePeriod=30
Oct 30 18:57:57.708990 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.708956    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.728952 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.728918    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:57:57.729161 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.729134    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kserve-container" containerID="cri-o://8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44" gracePeriod=30
Oct 30 18:57:57.729231 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.729184    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kube-rbac-proxy" containerID="cri-o://ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb" gracePeriod=30
Oct 30 18:57:57.849781 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.849747    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.976168 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:57.976144    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:57:58.051511 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.051471    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\" (UniqueName: \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\") pod \"11a7d2fb-7380-432c-a4eb-2777f35150cb\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") "
Oct 30 18:57:58.051686 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.051547    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\" (UniqueName: \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\") pod \"11a7d2fb-7380-432c-a4eb-2777f35150cb\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") "
Oct 30 18:57:58.053664 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.053625    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls" pod "11a7d2fb-7380-432c-a4eb-2777f35150cb" (UID: "11a7d2fb-7380-432c-a4eb-2777f35150cb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:57:58.053801 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.053673    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm" pod "11a7d2fb-7380-432c-a4eb-2777f35150cb" (UID: "11a7d2fb-7380-432c-a4eb-2777f35150cb"). InnerVolumeSpecName "kube-api-access-8vknm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:57:58.152387 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.152296    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\") pod \"11a7d2fb-7380-432c-a4eb-2777f35150cb\" (UID: \"11a7d2fb-7380-432c-a4eb-2777f35150cb\") "
Oct 30 18:57:58.152756 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.152432    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11a7d2fb-7380-432c-a4eb-2777f35150cb-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:57:58.152756 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.152443    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vknm\" (UniqueName: \"kubernetes.io/projected/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-api-access-8vknm\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:57:58.152756 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.152628    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config" pod "11a7d2fb-7380-432c-a4eb-2777f35150cb" (UID: "11a7d2fb-7380-432c-a4eb-2777f35150cb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:57:58.253311 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.253270    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/11a7d2fb-7380-432c-a4eb-2777f35150cb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:57:58.510483 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.510397    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerID="59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e" exitCode=0
Oct 30 18:57:58.510483 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.510467    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerDied","Data":"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"}
Oct 30 18:57:58.512058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512032    4938 generic.go:356] "Generic (PLEG): container finished" podID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerID="ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb" exitCode=0
Oct 30 18:57:58.512058 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512056    4938 generic.go:356] "Generic (PLEG): container finished" podID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerID="8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44" exitCode=2
Oct 30 18:57:58.512276 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512064    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerDied","Data":"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"}
Oct 30 18:57:58.512276 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512111    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"
Oct 30 18:57:58.512276 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512131    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerDied","Data":"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"}
Oct 30 18:57:58.512276 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512165    4938 scope.go:122] "RemoveContainer" containerID="ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"
Oct 30 18:57:58.512276 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.512166    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92" event={"ID":"11a7d2fb-7380-432c-a4eb-2777f35150cb","Type":"ContainerDied","Data":"38f671a85ff442f8327aa31dfab18e7420610c3eff84d436f5e418085b64bfdf"}
Oct 30 18:57:58.520258 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.520232    4938 scope.go:122] "RemoveContainer" containerID="8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"
Oct 30 18:57:58.528135 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.528111    4938 scope.go:122] "RemoveContainer" containerID="ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"
Oct 30 18:57:58.528530 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:57:58.528493    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb\": container with ID starting with ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb not found: ID does not exist" containerID="ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"
Oct 30 18:57:58.528611 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.528543    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"} err="failed to get container status \"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb\": rpc error: code = NotFound desc = could not find container \"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb\": container with ID starting with ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb not found: ID does not exist"
Oct 30 18:57:58.528611 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.528567    4938 scope.go:122] "RemoveContainer" containerID="8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"
Oct 30 18:57:58.528923 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:57:58.528898    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44\": container with ID starting with 8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44 not found: ID does not exist" containerID="8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"
Oct 30 18:57:58.528970 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.528933    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"} err="failed to get container status \"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44\": rpc error: code = NotFound desc = could not find container \"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44\": container with ID starting with 8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44 not found: ID does not exist"
Oct 30 18:57:58.528970 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.528951    4938 scope.go:122] "RemoveContainer" containerID="ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"
Oct 30 18:57:58.529213 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.529193    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb"} err="failed to get container status \"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb\": rpc error: code = NotFound desc = could not find container \"ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb\": container with ID starting with ea947cc731574ece33281dec0b1251df131a71b510635e4e242c409dadedffeb not found: ID does not exist"
Oct 30 18:57:58.529278 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.529214    4938 scope.go:122] "RemoveContainer" containerID="8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"
Oct 30 18:57:58.529490 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.529473    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44"} err="failed to get container status \"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44\": rpc error: code = NotFound desc = could not find container \"8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44\": container with ID starting with 8d6d22abe17741bdcd730d397fb7a80ac0e5ad2a863183b410f8d3d99dd62a44 not found: ID does not exist"
Oct 30 18:57:58.566029 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.565993    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:57:58.579394 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:57:58.579366    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/message-dumper-raw-6a446-predictor-6976756b99-mpl92"]
Oct 30 18:58:00.083104 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:00.083063    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:00.392678 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:00.392588    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" path="/var/lib/kubelet/pods/11a7d2fb-7380-432c-a4eb-2777f35150cb/volumes"
Oct 30 18:58:02.550366 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:02.550321    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:58:03.537269 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:03.537224    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerID="4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf" exitCode=0
Oct 30 18:58:03.537452 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:03.537298    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerDied","Data":"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"}
Oct 30 18:58:05.083667 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:05.083624    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:05.087342 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:05.087310    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:58:05.087670 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:05.087641    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:07.285935 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:07.285891    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:58:10.083045 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:10.082995    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:10.083518 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:10.083160    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:58:10.388165 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:10.388069    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.39:8080: connect: connection refused"
Oct 30 18:58:10.847462 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:10.847427    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:15.083754 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:15.083697    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:15.088128 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:15.088095    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:58:15.088519 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:15.088491    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:20.082894 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:20.082847    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:20.392224 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:20.392148    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:58:24.676830 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:24.676797    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:58:24.677241 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:24.677023    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container" containerID="cri-o://661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79" gracePeriod=30
Oct 30 18:58:24.677241 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:24.677097    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kube-rbac-proxy" containerID="cri-o://bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc" gracePeriod=30
Oct 30 18:58:24.746644 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:24.746608    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:25.083098 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.083048    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.36:8643/healthz\": dial tcp 10.133.0.36:8643: connect: connection refused"
Oct 30 18:58:25.087492 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.087457    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.36:8080: connect: connection refused"
Oct 30 18:58:25.087640 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.087617    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:58:25.088073 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.088047    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:58:25.088167 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.088156    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:58:25.630461 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.630429    4938 generic.go:356] "Generic (PLEG): container finished" podID="56899e54-54ed-407f-8adf-2e07fd2355de" containerID="bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc" exitCode=0
Oct 30 18:58:25.630644 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:25.630499    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerDied","Data":"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"}
Oct 30 18:58:27.280152 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.280105    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.39:8643/healthz\": dial tcp 10.133.0.39:8643: connect: connection refused"
Oct 30 18:58:27.792640 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.792613    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:58:27.898394 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898297    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\") pod \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") "
Oct 30 18:58:27.898394 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898383    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\") pod \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") "
Oct 30 18:58:27.898638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898406    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\") pod \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") "
Oct 30 18:58:27.898638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898440    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\" (UniqueName: \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\") pod \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") "
Oct 30 18:58:27.898638 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898494    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\") pod \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\" (UID: \"38d423c7-4fa3-4f9d-8df5-763eba9b8643\") "
Oct 30 18:58:27.898811 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898720    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location" pod "38d423c7-4fa3-4f9d-8df5-763eba9b8643" (UID: "38d423c7-4fa3-4f9d-8df5-763eba9b8643"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:27.898811 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898720    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config" pod "38d423c7-4fa3-4f9d-8df5-763eba9b8643" (UID: "38d423c7-4fa3-4f9d-8df5-763eba9b8643"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:27.898811 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.898788    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert" pod "38d423c7-4fa3-4f9d-8df5-763eba9b8643" (UID: "38d423c7-4fa3-4f9d-8df5-763eba9b8643"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:27.900651 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.900597    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls" pod "38d423c7-4fa3-4f9d-8df5-763eba9b8643" (UID: "38d423c7-4fa3-4f9d-8df5-763eba9b8643"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:27.900764 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.900674    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh" pod "38d423c7-4fa3-4f9d-8df5-763eba9b8643" (UID: "38d423c7-4fa3-4f9d-8df5-763eba9b8643"). InnerVolumeSpecName "kube-api-access-m5skh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:27.999317 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.999272    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:27.999317 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.999309    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d423c7-4fa3-4f9d-8df5-763eba9b8643-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:27.999317 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.999322    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:27.999559 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.999335    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5skh\" (UniqueName: \"kubernetes.io/projected/38d423c7-4fa3-4f9d-8df5-763eba9b8643-kube-api-access-m5skh\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:27.999559 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:27.999349    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d423c7-4fa3-4f9d-8df5-763eba9b8643-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:28.645456 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.645412    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerID="ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975" exitCode=0
Oct 30 18:58:28.645880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.645476    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerDied","Data":"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"}
Oct 30 18:58:28.645880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.645488    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"
Oct 30 18:58:28.645880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.645532    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl" event={"ID":"38d423c7-4fa3-4f9d-8df5-763eba9b8643","Type":"ContainerDied","Data":"6d3b86cd69ff9964c1858070ad1027408d1c590498932977bc7248c77f911079"}
Oct 30 18:58:28.645880 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.645571    4938 scope.go:122] "RemoveContainer" containerID="ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"
Oct 30 18:58:28.653726 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.653705    4938 scope.go:122] "RemoveContainer" containerID="59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"
Oct 30 18:58:28.661426 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.661398    4938 scope.go:122] "RemoveContainer" containerID="4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"
Oct 30 18:58:28.669269 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.669230    4938 scope.go:122] "RemoveContainer" containerID="26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"
Oct 30 18:58:28.677345 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.677320    4938 scope.go:122] "RemoveContainer" containerID="ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"
Oct 30 18:58:28.677629 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:28.677608    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975\": container with ID starting with ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975 not found: ID does not exist" containerID="ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"
Oct 30 18:58:28.677681 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.677641    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975"} err="failed to get container status \"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975\": rpc error: code = NotFound desc = could not find container \"ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975\": container with ID starting with ee3645d68def987b05c3aa6b6753afe201a12700d81e57a98eb17fa791188975 not found: ID does not exist"
Oct 30 18:58:28.677681 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.677658    4938 scope.go:122] "RemoveContainer" containerID="59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"
Oct 30 18:58:28.677907 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:28.677889    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e\": container with ID starting with 59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e not found: ID does not exist" containerID="59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"
Oct 30 18:58:28.677950 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.677914    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e"} err="failed to get container status \"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e\": rpc error: code = NotFound desc = could not find container \"59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e\": container with ID starting with 59c60406185ef415fe3e8a97e51ad05597071e18e41a0351e982fb11eb60571e not found: ID does not exist"
Oct 30 18:58:28.677950 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.677928    4938 scope.go:122] "RemoveContainer" containerID="4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"
Oct 30 18:58:28.678146 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:28.678129    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf\": container with ID starting with 4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf not found: ID does not exist" containerID="4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"
Oct 30 18:58:28.678186 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.678150    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf"} err="failed to get container status \"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf\": rpc error: code = NotFound desc = could not find container \"4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf\": container with ID starting with 4c45018a501ab36a6525e4f75dd9c0795b93628f088ed9584828578732959dbf not found: ID does not exist"
Oct 30 18:58:28.678186 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.678162    4938 scope.go:122] "RemoveContainer" containerID="26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"
Oct 30 18:58:28.678403 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:28.678388    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355\": container with ID starting with 26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355 not found: ID does not exist" containerID="26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"
Oct 30 18:58:28.678450 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.678410    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355"} err="failed to get container status \"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355\": rpc error: code = NotFound desc = could not find container \"26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355\": container with ID starting with 26d9a02851521c92b32e4cfbd9e2a9dbfc70db526f6cf29694329f843f641355 not found: ID does not exist"
Oct 30 18:58:28.713625 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.713593    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:58:28.726943 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:28.726912    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-logger-raw-6a446-predictor-7fdbcc645f-5kwxl"]
Oct 30 18:58:29.507271 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.507218    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.671036 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.670899    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671273    4938 cpu_manager.go:414] "containerMap: removing container" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671289    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671304    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="storage-initializer"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671309    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="storage-initializer"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671320    4938 cpu_manager.go:414] "containerMap: removing container" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kube-rbac-proxy"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671325    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kube-rbac-proxy"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671335    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671340    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671349    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671353    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671362    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671366    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671435    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671444    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="11a7d2fb-7380-432c-a4eb-2777f35150cb" containerName="kube-rbac-proxy"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671450    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kserve-container"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671455    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="agent"
Oct 30 18:58:29.671478 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.671465    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" containerName="kube-rbac-proxy"
Oct 30 18:58:29.675820 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.675804    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.704849 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704817    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704884    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-scale-raw-f5503-predictor-serving-cert\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704893    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-sklearn-scale-raw-f5503-kube-rbac-proxy-sar-config\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704891    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704909    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704900    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 18:58:29.705019 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.704931    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 18:58:29.706280 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.706261    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 18:58:29.728947 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.728920    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:29.814797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.814701    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.814797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.814743    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.814797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.814775    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.815017 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.814872    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5dkj\" (UniqueName: \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.815017 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.814912    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.915991 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.915949    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.915991 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.915995    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916225 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916029    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916225 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916065    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5dkj\" (UniqueName: \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916225 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916092    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916574 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916549    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916782 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916760    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.916850 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.916797    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.918492 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.918472    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.936752 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.936718    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5dkj\" (UniqueName: \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\") pod \"isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") " pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:29.992796 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:29.992759    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:30.032604 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.032571    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:58:30.118207 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118161    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\") pod \"56899e54-54ed-407f-8adf-2e07fd2355de\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") "
Oct 30 18:58:30.118467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118239    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\") pod \"56899e54-54ed-407f-8adf-2e07fd2355de\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") "
Oct 30 18:58:30.118467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118302    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\" (UniqueName: \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\") pod \"56899e54-54ed-407f-8adf-2e07fd2355de\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") "
Oct 30 18:58:30.118467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118334    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\") pod \"56899e54-54ed-407f-8adf-2e07fd2355de\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") "
Oct 30 18:58:30.118467 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118425    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\" (UniqueName: \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\") pod \"56899e54-54ed-407f-8adf-2e07fd2355de\" (UID: \"56899e54-54ed-407f-8adf-2e07fd2355de\") "
Oct 30 18:58:30.118711 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118584    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location" pod "56899e54-54ed-407f-8adf-2e07fd2355de" (UID: "56899e54-54ed-407f-8adf-2e07fd2355de"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:58:30.118711 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118672    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/56899e54-54ed-407f-8adf-2e07fd2355de-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:30.118797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118706    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config" pod "56899e54-54ed-407f-8adf-2e07fd2355de" (UID: "56899e54-54ed-407f-8adf-2e07fd2355de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:30.118797 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.118747    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert" pod "56899e54-54ed-407f-8adf-2e07fd2355de" (UID: "56899e54-54ed-407f-8adf-2e07fd2355de"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:58:30.120463 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.120436    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls" pod "56899e54-54ed-407f-8adf-2e07fd2355de" (UID: "56899e54-54ed-407f-8adf-2e07fd2355de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:58:30.120519 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.120495    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw" pod "56899e54-54ed-407f-8adf-2e07fd2355de" (UID: "56899e54-54ed-407f-8adf-2e07fd2355de"). InnerVolumeSpecName "kube-api-access-rhpdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:58:30.219927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.219889    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/56899e54-54ed-407f-8adf-2e07fd2355de-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:30.219927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.219921    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:30.219927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.219931    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhpdw\" (UniqueName: \"kubernetes.io/projected/56899e54-54ed-407f-8adf-2e07fd2355de-kube-api-access-rhpdw\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:30.219927 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.219940    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/56899e54-54ed-407f-8adf-2e07fd2355de-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 18:58:30.309857 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.309826    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 18:58:30.313068 ip-10-0-132-168 kubenswrapper[4938]: W1030 18:58:30.313036    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0017236_2e4e_4b90_9e84_e15e1281e34a.slice/crio-fc79f51ed88e8334f924b8e642bc8c4d4c2adbc5c86008f88d8a305baaa5c330 WatchSource:0}: Error finding container fc79f51ed88e8334f924b8e642bc8c4d4c2adbc5c86008f88d8a305baaa5c330: Status 404 returned error can't find the container with id fc79f51ed88e8334f924b8e642bc8c4d4c2adbc5c86008f88d8a305baaa5c330
Oct 30 18:58:30.392036 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.392003    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38d423c7-4fa3-4f9d-8df5-763eba9b8643" path="/var/lib/kubelet/pods/38d423c7-4fa3-4f9d-8df5-763eba9b8643/volumes"
Oct 30 18:58:30.658318 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.658210    4938 generic.go:356] "Generic (PLEG): container finished" podID="56899e54-54ed-407f-8adf-2e07fd2355de" containerID="661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79" exitCode=0
Oct 30 18:58:30.658318 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.658290    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerDied","Data":"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"}
Oct 30 18:58:30.658545 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.658315    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"
Oct 30 18:58:30.658545 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.658358    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f" event={"ID":"56899e54-54ed-407f-8adf-2e07fd2355de","Type":"ContainerDied","Data":"f343b7587ad2b3813584f2cb467107f0e34536b8c6508b06fc41050137729cd7"}
Oct 30 18:58:30.658545 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.658379    4938 scope.go:122] "RemoveContainer" containerID="bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"
Oct 30 18:58:30.660551 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.660488    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerStarted","Data":"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"}
Oct 30 18:58:30.660551 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.660549    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerStarted","Data":"fc79f51ed88e8334f924b8e642bc8c4d4c2adbc5c86008f88d8a305baaa5c330"}
Oct 30 18:58:30.668230 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.668210    4938 scope.go:122] "RemoveContainer" containerID="661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"
Oct 30 18:58:30.676119 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.676095    4938 scope.go:122] "RemoveContainer" containerID="331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"
Oct 30 18:58:30.685237 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.685216    4938 scope.go:122] "RemoveContainer" containerID="bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"
Oct 30 18:58:30.685590 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:30.685565    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc\": container with ID starting with bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc not found: ID does not exist" containerID="bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"
Oct 30 18:58:30.685706 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.685599    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc"} err="failed to get container status \"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc\": rpc error: code = NotFound desc = could not find container \"bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc\": container with ID starting with bb4b81c3dd13febef00d1cd82775c30f048391cc58dda3721b7f4feb11a6dafc not found: ID does not exist"
Oct 30 18:58:30.685706 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.685621    4938 scope.go:122] "RemoveContainer" containerID="661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"
Oct 30 18:58:30.685880 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:30.685862    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79\": container with ID starting with 661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79 not found: ID does not exist" containerID="661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"
Oct 30 18:58:30.685945 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.685883    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79"} err="failed to get container status \"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79\": rpc error: code = NotFound desc = could not find container \"661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79\": container with ID starting with 661d798691370a2472c57758287c95b3736ffca8d9412a616adcd28d826fed79 not found: ID does not exist"
Oct 30 18:58:30.685945 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.685899    4938 scope.go:122] "RemoveContainer" containerID="331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"
Oct 30 18:58:30.686129 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:30.686114    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af\": container with ID starting with 331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af not found: ID does not exist" containerID="331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"
Oct 30 18:58:30.686190 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.686131    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af"} err="failed to get container status \"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af\": rpc error: code = NotFound desc = could not find container \"331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af\": container with ID starting with 331a1b46085d1934a02e7ae85aa3079358b8295020263512e174e8a4a06058af not found: ID does not exist"
Oct 30 18:58:30.756532 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.756486    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:58:30.760695 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:30.760660    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-a3b71-predictor-78c494cb9c-vmd4f"]
Oct 30 18:58:32.393472 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:32.393435    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" path="/var/lib/kubelet/pods/56899e54-54ed-407f-8adf-2e07fd2355de/volumes"
Oct 30 18:58:32.550457 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:58:32.550415    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:58:34.678005 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:34.677972    4938 generic.go:356] "Generic (PLEG): container finished" podID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerID="50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893" exitCode=0
Oct 30 18:58:34.678423 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:34.678047    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerDied","Data":"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"}
Oct 30 18:58:35.684047 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:35.684013    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerStarted","Data":"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"}
Oct 30 18:58:35.684471 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:35.684079    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerStarted","Data":"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"}
Oct 30 18:58:35.684471 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:35.684287    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:35.742027 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:35.741970    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podStartSLOduration=2.892060826 podStartE2EDuration="6.741955282s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.849894456s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:35.741821139 +0000 UTC m=+1277.501829613" watchObservedRunningTime="2026-10-30 18:58:35.741955282 +0000 UTC m=+1277.501963742"
Oct 30 18:58:36.687886 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:36.687848    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:36.689102 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:36.689074    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:58:37.692165 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:37.692123    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:58:42.696520 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:42.696488    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 18:58:42.697178 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:42.697148    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:58:52.697028 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:58:52.696989    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:02.550146 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:59:02.550050    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:59:02.697218 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:02.697175    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:10.941993 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:10.941953    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:12.697680 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:12.697640    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:22.697160 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:22.697108    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:32.549932 ip-10-0-132-168 kubenswrapper[4938]: E1030 18:59:32.549889    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 18:59:32.697506 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:32.697462    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:42.697737 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:42.697688    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:52.387955 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:52.387908    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 18:59:59.722567 ip-10-0-132-168 kubenswrapper[4938]: I1030 18:59:59.722532    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:00.444759 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:00.444722    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:02.388105 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:02.388053    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:02.549323 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:00:02.549284    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:00:11.066808 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:11.066772    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:12.388114 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:12.388069    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:22.388440 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:22.388393    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:32.388264 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:32.388137    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:32.550057 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:00:32.550017    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:00:42.387898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:42.387857    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:44.365649 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.365612    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:44.582298 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582263    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:00:44.582619 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582606    4938 cpu_manager.go:414] "containerMap: removing container" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container"
Oct 30 19:00:44.582665 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582622    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container"
Oct 30 19:00:44.582665 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582635    4938 cpu_manager.go:414] "containerMap: removing container" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="storage-initializer"
Oct 30 19:00:44.582665 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582640    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="storage-initializer"
Oct 30 19:00:44.582766 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582676    4938 cpu_manager.go:414] "containerMap: removing container" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kube-rbac-proxy"
Oct 30 19:00:44.582766 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582682    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kube-rbac-proxy"
Oct 30 19:00:44.582766 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582731    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kube-rbac-proxy"
Oct 30 19:00:44.582766 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.582740    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="56899e54-54ed-407f-8adf-2e07fd2355de" containerName="kserve-container"
Oct 30 19:00:44.585828 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.585811    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.603669 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.603643    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"kube-root-ca.crt\""
Oct 30 19:00:44.616639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616569    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"default-dockercfg-gh4qc\""
Oct 30 19:00:44.616639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616572    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 19:00:44.616639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616601    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"openshift-service-ca.crt\""
Oct 30 19:00:44.616639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616617    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"raw-sklearn-runtime-b44ee-predictor-serving-cert\""
Oct 30 19:00:44.616639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616602    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"raw-sklearn-runtime-b44ee-kube-rbac-proxy-sar-config\""
Oct 30 19:00:44.616870 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.616643    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-d1ae4027ce\"/\"seaweedfs-s3-creds\""
Oct 30 19:00:44.622408 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.622380    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:00:44.645973 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.645941    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.646130 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.645992    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.646130 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.646086    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.646130 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.646123    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k59qg\" (UniqueName: \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.646281 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.646209    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747082 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747044    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747082 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747100    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k59qg\" (UniqueName: \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747398 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747145    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747398 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747179    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747398 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747238    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747556 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747516    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747833 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747812    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.747833 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.747823    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.749706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.749683    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.770560 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.770526    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k59qg\" (UniqueName: \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\") pod \"raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") " pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:44.902077 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:44.901988    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:45.217617 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:45.217541    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:00:45.220851 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:00:45.220822    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbe3c876_c448_4903_977b_572fba4bcae1.slice/crio-0314ce028c8029c2b3c239737cade0a53d2492ba7069ff185928ee1e4915debe WatchSource:0}: Error finding container 0314ce028c8029c2b3c239737cade0a53d2492ba7069ff185928ee1e4915debe: Status 404 returned error can't find the container with id 0314ce028c8029c2b3c239737cade0a53d2492ba7069ff185928ee1e4915debe
Oct 30 19:00:45.975581 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:45.975544    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:46.203646 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:46.203607    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerStarted","Data":"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"}
Oct 30 19:00:46.203830 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:46.203666    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerStarted","Data":"0314ce028c8029c2b3c239737cade0a53d2492ba7069ff185928ee1e4915debe"}
Oct 30 19:00:49.216657 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:49.216627    4938 generic.go:356] "Generic (PLEG): container finished" podID="fbe3c876-c448-4903-977b-572fba4bcae1" containerID="ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93" exitCode=0
Oct 30 19:00:49.217071 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:49.216663    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerDied","Data":"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"}
Oct 30 19:00:50.223285 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:50.223229    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerStarted","Data":"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"}
Oct 30 19:00:50.223749 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:50.223310    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerStarted","Data":"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"}
Oct 30 19:00:50.223749 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:50.223512    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:50.282877 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:50.282818    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podStartSLOduration=2.518051396 podStartE2EDuration="6.282803114s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.764751718s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:50.281714037 +0000 UTC m=+1412.041722502" watchObservedRunningTime="2026-10-30 19:00:50.282803114 +0000 UTC m=+1412.042811577"
Oct 30 19:00:51.228060 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:51.228019    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:51.229454 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:51.229427    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:00:52.232384 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:52.232345    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:00:52.388130 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:52.388087    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:00:57.237180 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:57.237147    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:00:57.237656 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:00:57.237625    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:02.388716 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:02.388673    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:01:02.549401 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:01:02.549365    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:01:03.387694 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:03.387653    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:01:07.237774 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:07.237728    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:11.198372 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:11.198329    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:13.387715 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:13.387657    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:01:17.237973 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:17.237925    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:23.389233 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:23.389195    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 19:01:27.237620 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:27.237576    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:27.885490 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:27.885453    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:30.650974 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:30.650929    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:30.793613 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:30.793574    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:31.079744 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.079708    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 19:01:31.080055 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.080025    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" containerID="cri-o://4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2" gracePeriod=30
Oct 30 19:01:31.080109 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.080087    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy" containerID="cri-o://bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d" gracePeriod=30
Oct 30 19:01:31.138969 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.138934    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:31.387985 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.387889    4938 generic.go:356] "Generic (PLEG): container finished" podID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerID="bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d" exitCode=0
Oct 30 19:01:31.387985 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:31.387945    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerDied","Data":"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"}
Oct 30 19:01:32.551152 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:01:32.551108    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:01:32.693266 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:32.693208    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 19:01:33.387631 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:33.387588    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.40:8080: connect: connection refused"
Oct 30 19:01:37.237994 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:37.237953    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:37.693405 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:37.693361    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.40:8643/healthz\": dial tcp 10.133.0.40:8643: connect: connection refused"
Oct 30 19:01:41.829715 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.829687    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 19:01:41.900637 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.900596    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\" (UniqueName: \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\") pod \"d0017236-2e4e-4b90-9e84-e15e1281e34a\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") "
Oct 30 19:01:41.900637 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.900641    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\") pod \"d0017236-2e4e-4b90-9e84-e15e1281e34a\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") "
Oct 30 19:01:41.900898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.900674    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\") pod \"d0017236-2e4e-4b90-9e84-e15e1281e34a\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") "
Oct 30 19:01:41.900898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.900704    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\") pod \"d0017236-2e4e-4b90-9e84-e15e1281e34a\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") "
Oct 30 19:01:41.900898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.900754    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\") pod \"d0017236-2e4e-4b90-9e84-e15e1281e34a\" (UID: \"d0017236-2e4e-4b90-9e84-e15e1281e34a\") "
Oct 30 19:01:41.901041 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.901019    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location" pod "d0017236-2e4e-4b90-9e84-e15e1281e34a" (UID: "d0017236-2e4e-4b90-9e84-e15e1281e34a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:41.901093 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.901063    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert" pod "d0017236-2e4e-4b90-9e84-e15e1281e34a" (UID: "d0017236-2e4e-4b90-9e84-e15e1281e34a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:41.901144 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.901123    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config" pod "d0017236-2e4e-4b90-9e84-e15e1281e34a" (UID: "d0017236-2e4e-4b90-9e84-e15e1281e34a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:41.903021 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.902991    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls" pod "d0017236-2e4e-4b90-9e84-e15e1281e34a" (UID: "d0017236-2e4e-4b90-9e84-e15e1281e34a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:41.903108 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:41.903065    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj" pod "d0017236-2e4e-4b90-9e84-e15e1281e34a" (UID: "d0017236-2e4e-4b90-9e84-e15e1281e34a"). InnerVolumeSpecName "kube-api-access-n5dkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:42.002323 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.002284    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5dkj\" (UniqueName: \"kubernetes.io/projected/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-api-access-n5dkj\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:01:42.002323 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.002318    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:01:42.002323 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.002327    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d0017236-2e4e-4b90-9e84-e15e1281e34a-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:01:42.002323 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.002337    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0017236-2e4e-4b90-9e84-e15e1281e34a-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:01:42.002587 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.002346    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d0017236-2e4e-4b90-9e84-e15e1281e34a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:01:42.433920 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.433882    4938 generic.go:356] "Generic (PLEG): container finished" podID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerID="4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2" exitCode=0
Oct 30 19:01:42.434119 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.433939    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerDied","Data":"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"}
Oct 30 19:01:42.434119 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.433963    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"
Oct 30 19:01:42.434119 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.434008    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7" event={"ID":"d0017236-2e4e-4b90-9e84-e15e1281e34a","Type":"ContainerDied","Data":"fc79f51ed88e8334f924b8e642bc8c4d4c2adbc5c86008f88d8a305baaa5c330"}
Oct 30 19:01:42.434119 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.434035    4938 scope.go:122] "RemoveContainer" containerID="bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"
Oct 30 19:01:42.442055 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.442037    4938 scope.go:122] "RemoveContainer" containerID="4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"
Oct 30 19:01:42.449269 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.449229    4938 scope.go:122] "RemoveContainer" containerID="50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"
Oct 30 19:01:42.458343 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.458321    4938 scope.go:122] "RemoveContainer" containerID="bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"
Oct 30 19:01:42.458626 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:01:42.458605    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d\": container with ID starting with bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d not found: ID does not exist" containerID="bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"
Oct 30 19:01:42.458689 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.458635    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d"} err="failed to get container status \"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d\": rpc error: code = NotFound desc = could not find container \"bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d\": container with ID starting with bf214e67a52a6e06340483fa9b7a824c2a0468f7fb6136655db3581fcc694d6d not found: ID does not exist"
Oct 30 19:01:42.458689 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.458655    4938 scope.go:122] "RemoveContainer" containerID="4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"
Oct 30 19:01:42.458938 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:01:42.458922    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2\": container with ID starting with 4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2 not found: ID does not exist" containerID="4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"
Oct 30 19:01:42.458989 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.458943    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2"} err="failed to get container status \"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2\": rpc error: code = NotFound desc = could not find container \"4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2\": container with ID starting with 4fdd297e5154ba5618bbccabf8b0878c2046c2330c2e4d6821b1d45f540771a2 not found: ID does not exist"
Oct 30 19:01:42.458989 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.458957    4938 scope.go:122] "RemoveContainer" containerID="50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"
Oct 30 19:01:42.459219 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:01:42.459197    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893\": container with ID starting with 50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893 not found: ID does not exist" containerID="50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"
Oct 30 19:01:42.459282 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.459226    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893"} err="failed to get container status \"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893\": rpc error: code = NotFound desc = could not find container \"50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893\": container with ID starting with 50751a32b56f9071eab8fd0f4199e243792c87c84086e25464306ea204cbf893 not found: ID does not exist"
Oct 30 19:01:42.501400 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.501366    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 19:01:42.514341 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:42.514309    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-sklearn-scale-raw-f5503-predictor-6646bcd9d8-j2gr7"]
Oct 30 19:01:44.392725 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.392695    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" path="/var/lib/kubelet/pods/d0017236-2e4e-4b90-9e84-e15e1281e34a/volumes"
Oct 30 19:01:44.724867 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.724788    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:44.826225 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826195    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:01:44.826508 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826495    4938 cpu_manager.go:414] "containerMap: removing container" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container"
Oct 30 19:01:44.826553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826509    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container"
Oct 30 19:01:44.826553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826522    4938 cpu_manager.go:414] "containerMap: removing container" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="storage-initializer"
Oct 30 19:01:44.826553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826528    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="storage-initializer"
Oct 30 19:01:44.826553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826540    4938 cpu_manager.go:414] "containerMap: removing container" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy"
Oct 30 19:01:44.826553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826546    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy"
Oct 30 19:01:44.826724 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826614    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kserve-container"
Oct 30 19:01:44.826724 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.826624    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0017236-2e4e-4b90-9e84-e15e1281e34a" containerName="kube-rbac-proxy"
Oct 30 19:01:44.831113 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.831090    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.861102 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861070    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 19:01:44.861102 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861070    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\""
Oct 30 19:01:44.861371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861115    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 19:01:44.861371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861149    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 19:01:44.861371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861210    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-create-v2-predictor-serving-cert\""
Oct 30 19:01:44.861371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861304    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 19:01:44.861635 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.861088    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 19:01:44.862512 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.862493    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:01:44.873594 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.873564    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:44.924390 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.924354    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzt96\" (UniqueName: \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.924598 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.924406    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.924598 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.924487    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.924598 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.924535    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.924730 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.924592    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:44.978872 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.978796    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:01:44.982147 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.982131    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:44.998060 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:44.998034    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-create-predictor-serving-cert\""
Oct 30 19:01:45.003773 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.003747    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:01:45.018160 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.018121    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:45.024987 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.024949    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.024987 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.024987    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.025228 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.025139    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzt96\" (UniqueName: \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.025228 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.025212    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.025392 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.025289    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.026015 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.025986    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.026280 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.026237    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.026364 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.026329    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.027645 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.027624    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.046034 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.046000    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzt96\" (UniqueName: \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\") pod \"isvc-canary-create-v2-predictor-5f49875c48-6zhv6\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") " pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.126278 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.126209    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.126278 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.126279    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.126494 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.126404    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.126494 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.126443    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqfb9\" (UniqueName: \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.126494 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.126487    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.148212 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.148172    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:45.161039 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.161009    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:45.226907 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.226876    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.226907 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.226912    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227099 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.226939    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227099 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.226960    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqfb9\" (UniqueName: \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227099 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.226993    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227655 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.227599    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227655 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.227611    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.227832 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.227813    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.229634 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.229580    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.247783 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.247758    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqfb9\" (UniqueName: \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\") pod \"isvc-canary-create-predictor-6c4b8d6c8c-7lj9r\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") " pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.298460 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.298418    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:45.459229 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.459205    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:01:45.461867 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:01:45.461838    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8548d9bb_5636_467f_aea0_f5a94c368b6e.slice/crio-5f4be74140cf4d8c7ca0aec05a46736b394cfc5debf02b4587a773b05b04e7fe WatchSource:0}: Error finding container 5f4be74140cf4d8c7ca0aec05a46736b394cfc5debf02b4587a773b05b04e7fe: Status 404 returned error can't find the container with id 5f4be74140cf4d8c7ca0aec05a46736b394cfc5debf02b4587a773b05b04e7fe
Oct 30 19:01:45.600960 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:45.600931    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:01:45.604372 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:01:45.604345    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813e7381_1726_499c_ba68_6aa7f84f7aaf.slice/crio-876e0464bbc517b080da142f5969c090684f64f9c9225efb6259ece879561432 WatchSource:0}: Error finding container 876e0464bbc517b080da142f5969c090684f64f9c9225efb6259ece879561432: Status 404 returned error can't find the container with id 876e0464bbc517b080da142f5969c090684f64f9c9225efb6259ece879561432
Oct 30 19:01:46.454840 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:46.454794    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerStarted","Data":"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"}
Oct 30 19:01:46.455027 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:46.454869    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerStarted","Data":"876e0464bbc517b080da142f5969c090684f64f9c9225efb6259ece879561432"}
Oct 30 19:01:46.456387 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:46.456359    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerStarted","Data":"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"}
Oct 30 19:01:46.456502 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:46.456416    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerStarted","Data":"5f4be74140cf4d8c7ca0aec05a46736b394cfc5debf02b4587a773b05b04e7fe"}
Oct 30 19:01:47.238393 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:47.238349    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:50.476188 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:50.476152    4938 generic.go:356] "Generic (PLEG): container finished" podID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerID="7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5" exitCode=0
Oct 30 19:01:50.476654 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:50.476226    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerDied","Data":"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"}
Oct 30 19:01:50.477960 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:50.477935    4938 generic.go:356] "Generic (PLEG): container finished" podID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerID="6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2" exitCode=0
Oct 30 19:01:50.478072 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:50.477963    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerDied","Data":"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"}
Oct 30 19:01:51.484026 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.483973    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerStarted","Data":"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"}
Oct 30 19:01:51.484520 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.484082    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerStarted","Data":"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"}
Oct 30 19:01:51.484520 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.484307    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:51.486025 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.485994    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerStarted","Data":"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"}
Oct 30 19:01:51.486174 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.486049    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerStarted","Data":"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"}
Oct 30 19:01:51.486319 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.486301    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:51.486375 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.486343    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:51.487791 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.487763    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:01:51.542581 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.542523    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podStartSLOduration=3.682349855 podStartE2EDuration="7.542510516s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.860160661s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:51.541295562 +0000 UTC m=+1473.301304021" watchObservedRunningTime="2026-10-30 19:01:51.542510516 +0000 UTC m=+1473.302518956"
Oct 30 19:01:51.586533 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:51.586474    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podStartSLOduration=3.5645817969999998 podStartE2EDuration="7.586460619s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.021878822s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:51.585454822 +0000 UTC m=+1473.345463286" watchObservedRunningTime="2026-10-30 19:01:51.586460619 +0000 UTC m=+1473.346469073"
Oct 30 19:01:52.490267 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:52.490220    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:52.490744 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:52.490239    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:01:52.491561 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:52.491536    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:01:53.495144 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:53.495100    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:01:57.238409 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:57.238370    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:01:57.494986 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:57.494900    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:01:57.495574 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:57.495549    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:01:58.499821 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:58.499791    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:01:58.500324 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:01:58.500298    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:02.549694 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:02:02.549597    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:02:03.388336 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:03.388292    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.41:8080: connect: connection refused"
Oct 30 19:02:07.496504 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:07.496457    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:08.501123 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:08.501079    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:11.303746 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:11.303708    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:13.388920 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:13.388885    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:02:15.177749 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.177708    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:02:15.178193 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.178011    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container" containerID="cri-o://3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415" gracePeriod=30
Oct 30 19:02:15.178193 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.178058    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kube-rbac-proxy" containerID="cri-o://8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a" gracePeriod=30
Oct 30 19:02:15.251393 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.251362    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:15.593024 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.592988    4938 generic.go:356] "Generic (PLEG): container finished" podID="fbe3c876-c448-4903-977b-572fba4bcae1" containerID="8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a" exitCode=0
Oct 30 19:02:15.593227 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:15.593051    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerDied","Data":"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"}
Oct 30 19:02:17.232799 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:17.232755    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.41:8643/healthz\": dial tcp 10.133.0.41:8643: connect: connection refused"
Oct 30 19:02:17.496543 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:17.496454    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:18.500827 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:18.500789    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:18.525278 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:18.525224    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 19:02:18.529705 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:18.529680    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 19:02:18.534954 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:18.534929    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 19:02:18.539482 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:18.539459    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 19:02:20.926735 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:20.926708    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:02:21.031609 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031511    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\") pod \"fbe3c876-c448-4903-977b-572fba4bcae1\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") "
Oct 30 19:02:21.031609 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031584    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\" (UniqueName: \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\") pod \"fbe3c876-c448-4903-977b-572fba4bcae1\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") "
Oct 30 19:02:21.031609 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031612    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\") pod \"fbe3c876-c448-4903-977b-572fba4bcae1\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") "
Oct 30 19:02:21.031898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031773    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\") pod \"fbe3c876-c448-4903-977b-572fba4bcae1\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") "
Oct 30 19:02:21.031898 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031834    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\") pod \"fbe3c876-c448-4903-977b-572fba4bcae1\" (UID: \"fbe3c876-c448-4903-977b-572fba4bcae1\") "
Oct 30 19:02:21.032004 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031925    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert" pod "fbe3c876-c448-4903-977b-572fba4bcae1" (UID: "fbe3c876-c448-4903-977b-572fba4bcae1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:21.032004 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.031933    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config" pod "fbe3c876-c448-4903-977b-572fba4bcae1" (UID: "fbe3c876-c448-4903-977b-572fba4bcae1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:02:21.032150 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.032133    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:02:21.032209 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.032160    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/fbe3c876-c448-4903-977b-572fba4bcae1-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:02:21.032209 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.032132    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location" pod "fbe3c876-c448-4903-977b-572fba4bcae1" (UID: "fbe3c876-c448-4903-977b-572fba4bcae1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:02:21.033801 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.033776    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls" pod "fbe3c876-c448-4903-977b-572fba4bcae1" (UID: "fbe3c876-c448-4903-977b-572fba4bcae1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:02:21.033801 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.033777    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg" pod "fbe3c876-c448-4903-977b-572fba4bcae1" (UID: "fbe3c876-c448-4903-977b-572fba4bcae1"). InnerVolumeSpecName "kube-api-access-k59qg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:02:21.133204 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.133146    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k59qg\" (UniqueName: \"kubernetes.io/projected/fbe3c876-c448-4903-977b-572fba4bcae1-kube-api-access-k59qg\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:02:21.133204 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.133201    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fbe3c876-c448-4903-977b-572fba4bcae1-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:02:21.133204 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.133212    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fbe3c876-c448-4903-977b-572fba4bcae1-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:02:21.618361 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.618326    4938 generic.go:356] "Generic (PLEG): container finished" podID="fbe3c876-c448-4903-977b-572fba4bcae1" containerID="3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415" exitCode=0
Oct 30 19:02:21.618510 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.618385    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"
Oct 30 19:02:21.618510 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.618397    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerDied","Data":"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"}
Oct 30 19:02:21.618510 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.618450    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt" event={"ID":"fbe3c876-c448-4903-977b-572fba4bcae1","Type":"ContainerDied","Data":"0314ce028c8029c2b3c239737cade0a53d2492ba7069ff185928ee1e4915debe"}
Oct 30 19:02:21.618510 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.618478    4938 scope.go:122] "RemoveContainer" containerID="8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"
Oct 30 19:02:21.627626 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.627608    4938 scope.go:122] "RemoveContainer" containerID="3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"
Oct 30 19:02:21.635133 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.635111    4938 scope.go:122] "RemoveContainer" containerID="ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"
Oct 30 19:02:21.643125 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.643099    4938 scope.go:122] "RemoveContainer" containerID="8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"
Oct 30 19:02:21.643540 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:02:21.643513    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a\": container with ID starting with 8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a not found: ID does not exist" containerID="8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"
Oct 30 19:02:21.643610 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.643553    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a"} err="failed to get container status \"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a\": rpc error: code = NotFound desc = could not find container \"8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a\": container with ID starting with 8d8d47b6baa9a8c4d09f1276467745667b047fd2dbde882f605378e715f4628a not found: ID does not exist"
Oct 30 19:02:21.643610 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.643577    4938 scope.go:122] "RemoveContainer" containerID="3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"
Oct 30 19:02:21.643879 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:02:21.643856    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415\": container with ID starting with 3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415 not found: ID does not exist" containerID="3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"
Oct 30 19:02:21.643926 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.643887    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415"} err="failed to get container status \"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415\": rpc error: code = NotFound desc = could not find container \"3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415\": container with ID starting with 3d28297325c07c126f579484d1daa9186664b82868a0e97bc27d6d0059555415 not found: ID does not exist"
Oct 30 19:02:21.643926 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.643907    4938 scope.go:122] "RemoveContainer" containerID="ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"
Oct 30 19:02:21.644194 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:02:21.644173    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93\": container with ID starting with ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93 not found: ID does not exist" containerID="ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"
Oct 30 19:02:21.644273 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.644201    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93"} err="failed to get container status \"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93\": rpc error: code = NotFound desc = could not find container \"ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93\": container with ID starting with ce0a60d3f413e09804237882a1f46a42a6c16f84f5f340348fd5227174557b93 not found: ID does not exist"
Oct 30 19:02:21.673418 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.673383    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:02:21.686554 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:21.686526    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-d1ae4027ce/raw-sklearn-runtime-b44ee-predictor-5949748865-b2bjt"]
Oct 30 19:02:22.392702 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:22.392667    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" path="/var/lib/kubelet/pods/fbe3c876-c448-4903-977b-572fba4bcae1/volumes"
Oct 30 19:02:27.496261 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:27.496200    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:28.500725 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:28.500682    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:32.548471 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:02:32.548433    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:02:37.496018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:37.495972    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:38.500992 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:38.500947    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:47.495812 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:47.495768    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:48.500474 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:48.500432    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:02:57.495865 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:57.495819    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:02:58.500649 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:02:58.500609    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:03:02.456124 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:02.456078    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:02.549778 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:02.549739    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:03:05.388073 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:05.388020    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:03:07.495596 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:07.495553    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.42:8080: connect: connection refused"
Oct 30 19:03:09.685842 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:09.685800    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:11.427217 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:11.427176    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:12.392453 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:12.392424    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:03:15.389379 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:15.389345    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:03:21.093311 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:21.093243    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:25.002269 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.002223    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:03:25.002749 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.002545    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container" containerID="cri-o://dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47" gracePeriod=30
Oct 30 19:03:25.002749 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.002629    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kube-rbac-proxy" containerID="cri-o://bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570" gracePeriod=30
Oct 30 19:03:25.005763 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.005738    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:03:25.005979 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.005960    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" containerID="cri-o://e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40" gracePeriod=30
Oct 30 19:03:25.006059 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.006034    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kube-rbac-proxy" containerID="cri-o://639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86" gracePeriod=30
Oct 30 19:03:25.068462 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.068428    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:25.212151 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.212116    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:25.388813 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.388773    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.43:8080: connect: connection refused"
Oct 30 19:03:25.879364 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.879322    4938 generic.go:356] "Generic (PLEG): container finished" podID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerID="639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86" exitCode=0
Oct 30 19:03:25.879543 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.879399    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerDied","Data":"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"}
Oct 30 19:03:25.881382 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.881358    4938 generic.go:356] "Generic (PLEG): container finished" podID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerID="bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570" exitCode=0
Oct 30 19:03:25.881486 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:25.881410    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerDied","Data":"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"}
Oct 30 19:03:27.491116 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:27.491061    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.42:8643/healthz\": dial tcp 10.133.0.42:8643: connect: connection refused"
Oct 30 19:03:28.495681 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:28.495626    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.43:8643/healthz\": dial tcp 10.133.0.43:8643: connect: connection refused"
Oct 30 19:03:30.358195 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.358167    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:03:30.460495 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.460469    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:03:30.491818 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.491782    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\") pod \"813e7381-1726-499c-ba68-6aa7f84f7aaf\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") "
Oct 30 19:03:30.492004 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.491871    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\") pod \"813e7381-1726-499c-ba68-6aa7f84f7aaf\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") "
Oct 30 19:03:30.492004 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.491967    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\") pod \"813e7381-1726-499c-ba68-6aa7f84f7aaf\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") "
Oct 30 19:03:30.492127 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492015    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\" (UniqueName: \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\") pod \"813e7381-1726-499c-ba68-6aa7f84f7aaf\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") "
Oct 30 19:03:30.492127 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492045    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\" (UniqueName: \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\") pod \"813e7381-1726-499c-ba68-6aa7f84f7aaf\" (UID: \"813e7381-1726-499c-ba68-6aa7f84f7aaf\") "
Oct 30 19:03:30.492236 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492122    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location" pod "813e7381-1726-499c-ba68-6aa7f84f7aaf" (UID: "813e7381-1726-499c-ba68-6aa7f84f7aaf"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:30.492310 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492239    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert" pod "813e7381-1726-499c-ba68-6aa7f84f7aaf" (UID: "813e7381-1726-499c-ba68-6aa7f84f7aaf"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:30.492369 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492340    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/813e7381-1726-499c-ba68-6aa7f84f7aaf-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.492369 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492341    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config" pod "813e7381-1726-499c-ba68-6aa7f84f7aaf" (UID: "813e7381-1726-499c-ba68-6aa7f84f7aaf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:30.492369 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.492358    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.494194 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.494163    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9" pod "813e7381-1726-499c-ba68-6aa7f84f7aaf" (UID: "813e7381-1726-499c-ba68-6aa7f84f7aaf"). InnerVolumeSpecName "kube-api-access-hqfb9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:30.494194 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.494169    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls" pod "813e7381-1726-499c-ba68-6aa7f84f7aaf" (UID: "813e7381-1726-499c-ba68-6aa7f84f7aaf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:30.592882 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.592848    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\" (UniqueName: \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\") pod \"8548d9bb-5636-467f-aea0-f5a94c368b6e\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") "
Oct 30 19:03:30.593057 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.592896    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\") pod \"8548d9bb-5636-467f-aea0-f5a94c368b6e\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") "
Oct 30 19:03:30.593057 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.592918    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\") pod \"8548d9bb-5636-467f-aea0-f5a94c368b6e\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") "
Oct 30 19:03:30.593057 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.592954    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\") pod \"8548d9bb-5636-467f-aea0-f5a94c368b6e\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") "
Oct 30 19:03:30.593057 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.592977    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\") pod \"8548d9bb-5636-467f-aea0-f5a94c368b6e\" (UID: \"8548d9bb-5636-467f-aea0-f5a94c368b6e\") "
Oct 30 19:03:30.593292 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593159    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.593292 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593179    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/813e7381-1726-499c-ba68-6aa7f84f7aaf-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.593292 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593194    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqfb9\" (UniqueName: \"kubernetes.io/projected/813e7381-1726-499c-ba68-6aa7f84f7aaf-kube-api-access-hqfb9\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.593461 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593305    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location" pod "8548d9bb-5636-467f-aea0-f5a94c368b6e" (UID: "8548d9bb-5636-467f-aea0-f5a94c368b6e"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:03:30.593461 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593337    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert" pod "8548d9bb-5636-467f-aea0-f5a94c368b6e" (UID: "8548d9bb-5636-467f-aea0-f5a94c368b6e"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:30.593461 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.593401    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config" pod "8548d9bb-5636-467f-aea0-f5a94c368b6e" (UID: "8548d9bb-5636-467f-aea0-f5a94c368b6e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:03:30.594982 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.594961    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96" pod "8548d9bb-5636-467f-aea0-f5a94c368b6e" (UID: "8548d9bb-5636-467f-aea0-f5a94c368b6e"). InnerVolumeSpecName "kube-api-access-wzt96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:03:30.595098 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.595079    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls" pod "8548d9bb-5636-467f-aea0-f5a94c368b6e" (UID: "8548d9bb-5636-467f-aea0-f5a94c368b6e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:03:30.694046 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.694000    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.694046 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.694037    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8548d9bb-5636-467f-aea0-f5a94c368b6e-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.694046 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.694050    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzt96\" (UniqueName: \"kubernetes.io/projected/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-api-access-wzt96\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.694313 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.694062    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8548d9bb-5636-467f-aea0-f5a94c368b6e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.694313 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.694075    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8548d9bb-5636-467f-aea0-f5a94c368b6e-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:03:30.904358 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.904319    4938 generic.go:356] "Generic (PLEG): container finished" podID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerID="e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40" exitCode=0
Oct 30 19:03:30.904534 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.904386    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"
Oct 30 19:03:30.904534 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.904405    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerDied","Data":"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"}
Oct 30 19:03:30.904534 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.904484    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r" event={"ID":"813e7381-1726-499c-ba68-6aa7f84f7aaf","Type":"ContainerDied","Data":"876e0464bbc517b080da142f5969c090684f64f9c9225efb6259ece879561432"}
Oct 30 19:03:30.904534 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.904523    4938 scope.go:122] "RemoveContainer" containerID="639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"
Oct 30 19:03:30.906500 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.906479    4938 generic.go:356] "Generic (PLEG): container finished" podID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerID="dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47" exitCode=0
Oct 30 19:03:30.906595 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.906510    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerDied","Data":"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"}
Oct 30 19:03:30.906595 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.906541    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"
Oct 30 19:03:30.906595 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.906577    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6" event={"ID":"8548d9bb-5636-467f-aea0-f5a94c368b6e","Type":"ContainerDied","Data":"5f4be74140cf4d8c7ca0aec05a46736b394cfc5debf02b4587a773b05b04e7fe"}
Oct 30 19:03:30.914673 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.914658    4938 scope.go:122] "RemoveContainer" containerID="e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"
Oct 30 19:03:30.922603 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.922584    4938 scope.go:122] "RemoveContainer" containerID="7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"
Oct 30 19:03:30.929764 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.929747    4938 scope.go:122] "RemoveContainer" containerID="639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"
Oct 30 19:03:30.930029 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.930008    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86\": container with ID starting with 639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86 not found: ID does not exist" containerID="639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"
Oct 30 19:03:30.930110 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930043    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86"} err="failed to get container status \"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86\": rpc error: code = NotFound desc = could not find container \"639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86\": container with ID starting with 639ea4ddbe0c0f36b42ab2947bdaec95bbb55e9ace6fb0562ceedfdbb2b3eb86 not found: ID does not exist"
Oct 30 19:03:30.930110 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930068    4938 scope.go:122] "RemoveContainer" containerID="e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"
Oct 30 19:03:30.930332 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.930315    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40\": container with ID starting with e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40 not found: ID does not exist" containerID="e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"
Oct 30 19:03:30.930390 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930339    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40"} err="failed to get container status \"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40\": rpc error: code = NotFound desc = could not find container \"e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40\": container with ID starting with e0fc10f5560029ae144792bcd44969c73874d3bbd882ef8520862a77d91e8b40 not found: ID does not exist"
Oct 30 19:03:30.930390 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930353    4938 scope.go:122] "RemoveContainer" containerID="7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"
Oct 30 19:03:30.930577 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.930559    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5\": container with ID starting with 7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5 not found: ID does not exist" containerID="7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"
Oct 30 19:03:30.930620 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930582    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5"} err="failed to get container status \"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5\": rpc error: code = NotFound desc = could not find container \"7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5\": container with ID starting with 7a97b025ec1ad526b25d9bbed4c5daa809cf84f7a3cc25facc0475a0aa4cebd5 not found: ID does not exist"
Oct 30 19:03:30.930620 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.930598    4938 scope.go:122] "RemoveContainer" containerID="bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"
Oct 30 19:03:30.938129 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.938111    4938 scope.go:122] "RemoveContainer" containerID="dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"
Oct 30 19:03:30.945096 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.945079    4938 scope.go:122] "RemoveContainer" containerID="6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"
Oct 30 19:03:30.952630 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.952615    4938 scope.go:122] "RemoveContainer" containerID="bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"
Oct 30 19:03:30.952898 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.952874    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570\": container with ID starting with bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570 not found: ID does not exist" containerID="bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"
Oct 30 19:03:30.952994 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.952903    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570"} err="failed to get container status \"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570\": rpc error: code = NotFound desc = could not find container \"bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570\": container with ID starting with bbdd69f5264d16e7aa45880648902d56d9c4db955a996bd0d06801d9d8369570 not found: ID does not exist"
Oct 30 19:03:30.952994 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.952925    4938 scope.go:122] "RemoveContainer" containerID="dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"
Oct 30 19:03:30.953169 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.953153    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47\": container with ID starting with dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47 not found: ID does not exist" containerID="dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"
Oct 30 19:03:30.953216 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.953174    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47"} err="failed to get container status \"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47\": rpc error: code = NotFound desc = could not find container \"dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47\": container with ID starting with dc8f650beed9aa0a0eb9b46050046f2dd7221603d9da039c4f940c2330956f47 not found: ID does not exist"
Oct 30 19:03:30.953216 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.953189    4938 scope.go:122] "RemoveContainer" containerID="6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"
Oct 30 19:03:30.953435 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:30.953417    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2\": container with ID starting with 6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2 not found: ID does not exist" containerID="6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"
Oct 30 19:03:30.953481 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.953442    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2"} err="failed to get container status \"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2\": rpc error: code = NotFound desc = could not find container \"6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2\": container with ID starting with 6c50fe516746c6d596a4c4df341ca24743065f22d21be4bc77e23c058b9882b2 not found: ID does not exist"
Oct 30 19:03:30.958529 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.958505    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:03:30.971481 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.971455    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-v2-predictor-5f49875c48-6zhv6"]
Oct 30 19:03:30.998584 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:30.998550    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:03:31.004321 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:31.004296    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-create-predictor-6c4b8d6c8c-7lj9r"]
Oct 30 19:03:32.392159 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.392122    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" path="/var/lib/kubelet/pods/813e7381-1726-499c-ba68-6aa7f84f7aaf/volumes"
Oct 30 19:03:32.392648 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.392633    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" path="/var/lib/kubelet/pods/8548d9bb-5636-467f-aea0-f5a94c368b6e/volumes"
Oct 30 19:03:32.550507 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:03:32.550466    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:03:32.613107 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.613072    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:32.756900 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.756815    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:32.816236 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816202    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:03:32.816529 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816515    4938 cpu_manager.go:414] "containerMap: removing container" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container"
Oct 30 19:03:32.816567 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816531    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container"
Oct 30 19:03:32.816567 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816540    4938 cpu_manager.go:414] "containerMap: removing container" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816567 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816546    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816567 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816555    4938 cpu_manager.go:414] "containerMap: removing container" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816567 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816559    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816571    4938 cpu_manager.go:414] "containerMap: removing container" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816576    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816583    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816588    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816594    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816599    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816611    4938 cpu_manager.go:414] "containerMap: removing container" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816615    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816626    4938 cpu_manager.go:414] "containerMap: removing container" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816631    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="storage-initializer"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816636    4938 cpu_manager.go:414] "containerMap: removing container" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816640    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816687    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816695    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816701    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbe3c876-c448-4903-977b-572fba4bcae1" containerName="kube-rbac-proxy"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816707    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816714    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="813e7381-1726-499c-ba68-6aa7f84f7aaf" containerName="kserve-container"
Oct 30 19:03:32.816719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.816719    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="8548d9bb-5636-467f-aea0-f5a94c368b6e" containerName="kube-rbac-proxy"
Oct 30 19:03:32.820942 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.820916    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.849896 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849855    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"kube-root-ca.crt\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849910    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849855    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849860    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"openshift-service-ca.crt\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849873    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"seaweedfs-s3-creds\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.849873    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Oct 30 19:03:32.850104 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.850102    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"default-dockercfg-c4gjk\""
Oct 30 19:03:32.851447 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.851423    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:03:32.864062 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.864028    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:03:32.867568 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.867547    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.880326 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.880298    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:03:32.903304 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.903267    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:32.911947 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.911919    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.911947 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.911953    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.912162 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.911991    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.912162 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912093    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.912162 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912132    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.912279 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912160    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxsf5\" (UniqueName: \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.912279 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912239    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvk6l\" (UniqueName: \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.912350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912296    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.912350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912324    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:32.912350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.912346    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:32.936779 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:32.936748    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-42351c4657\"/\"isvc-canary-promote-predictor-serving-cert\""
Oct 30 19:03:33.013086 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013000    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013086 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013049    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.013086 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013078    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.013396 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013109    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013396 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013139    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013396 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013176    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013396 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013208    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxsf5\" (UniqueName: \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013606 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013506    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvk6l\" (UniqueName: \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.013606 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013518    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013606 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013558    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.013606 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013598    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.013951 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013929    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.013951 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013942    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.014078 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013962    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.014078 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.013988    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.014298 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.014279    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.015795 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.015771    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.015895 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.015772    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.034876 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.034844    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxsf5\" (UniqueName: \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\") pod \"isvc-canary-promote-v2-predictor-dc54999c4-spmt5\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.034999 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.034965    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvk6l\" (UniqueName: \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\") pod \"isvc-canary-promote-predictor-b9c5956c9-kkwzh\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") " pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.141693 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.141648    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:33.184962 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.184923    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:33.454777 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.454749    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:03:33.457576 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:03:33.457551    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19157767_3c2a_42e0_af2b_852a9d613136.slice/crio-bb5008abeb4bb3bfba222fe70041dada8e55e8237889cfa7a44973bcbc5ee708 WatchSource:0}: Error finding container bb5008abeb4bb3bfba222fe70041dada8e55e8237889cfa7a44973bcbc5ee708: Status 404 returned error can't find the container with id bb5008abeb4bb3bfba222fe70041dada8e55e8237889cfa7a44973bcbc5ee708
Oct 30 19:03:33.493879 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.493844    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:03:33.497292 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:03:33.497261    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d064b5_befd_4324_b5d3_94dbfacd5d02.slice/crio-efe26b68c87388dea00eaa368e1ba2343c7d7a53e4e65fe9a761b7ae4dc7d461 WatchSource:0}: Error finding container efe26b68c87388dea00eaa368e1ba2343c7d7a53e4e65fe9a761b7ae4dc7d461: Status 404 returned error can't find the container with id efe26b68c87388dea00eaa368e1ba2343c7d7a53e4e65fe9a761b7ae4dc7d461
Oct 30 19:03:33.923058 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.923019    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerStarted","Data":"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"}
Oct 30 19:03:33.923311 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.923098    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerStarted","Data":"efe26b68c87388dea00eaa368e1ba2343c7d7a53e4e65fe9a761b7ae4dc7d461"}
Oct 30 19:03:33.924810 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.924785    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerStarted","Data":"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"}
Oct 30 19:03:33.924923 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:33.924828    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerStarted","Data":"bb5008abeb4bb3bfba222fe70041dada8e55e8237889cfa7a44973bcbc5ee708"}
Oct 30 19:03:37.941327 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:37.941284    4938 generic.go:356] "Generic (PLEG): container finished" podID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerID="35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6" exitCode=0
Oct 30 19:03:37.941804 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:37.941355    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerDied","Data":"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"}
Oct 30 19:03:37.943066 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:37.943039    4938 generic.go:356] "Generic (PLEG): container finished" podID="19157767-3c2a-42e0-af2b-852a9d613136" containerID="6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a" exitCode=0
Oct 30 19:03:37.943161 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:37.943075    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerDied","Data":"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"}
Oct 30 19:03:38.949201 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.949167    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerStarted","Data":"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"}
Oct 30 19:03:38.949719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.949239    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerStarted","Data":"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"}
Oct 30 19:03:38.949719 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.949450    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:38.951300 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.951276    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerStarted","Data":"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"}
Oct 30 19:03:38.951417 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.951324    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerStarted","Data":"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"}
Oct 30 19:03:38.951517 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.951499    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:38.951554 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.951545    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:38.952973 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:38.952948    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:03:39.007955 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:39.007901    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podStartSLOduration=3.189302226 podStartE2EDuration="7.007888374s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.818586148s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:39.007191847 +0000 UTC m=+1580.767200309" watchObservedRunningTime="2026-10-30 19:03:39.007888374 +0000 UTC m=+1580.767896833"
Oct 30 19:03:39.054653 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:39.054594    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podStartSLOduration=3.120711084 podStartE2EDuration="7.054576641s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.933865557s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:39.053547588 +0000 UTC m=+1580.813556050" watchObservedRunningTime="2026-10-30 19:03:39.054576641 +0000 UTC m=+1580.814585103"
Oct 30 19:03:39.957426 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:39.957346    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:39.957426 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:39.957366    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:03:39.958888 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:39.958860    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:03:40.961059 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:40.961016    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:03:44.962591 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:44.962555    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:03:44.963330 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:44.963302    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:03:45.965872 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:45.965840    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:03:45.966521 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:45.966492    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:03:54.964176 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:54.964127    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:03:55.967378 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:03:55.967334    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:02.549239 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:04:02.549195    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:04:04.963356 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:04.963310    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:05.967169 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:05.967129    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:12.878166 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:12.878123    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:14.963731 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:14.963689    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:15.967269 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:15.967219    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:24.963350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:24.963308    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:25.012507 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:25.012457    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:25.967409 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:25.967364    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:31.450568 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:31.450523    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:31.609178 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:31.609139    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:32.550377 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:04:32.550338    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:04:34.963437 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:34.963398    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:35.967053 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:35.967005    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:44.963468 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:44.963428    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:45.966419 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:45.966382    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:49.387858 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:49.387817    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.45:8080: connect: connection refused"
Oct 30 19:04:50.388239 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:50.388189    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.44:8080: connect: connection refused"
Oct 30 19:04:59.388967 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:04:59.388932    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:05:00.392495 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:00.392465    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:05:02.550683 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:05:02.550641    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:05:03.023306 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.023270    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:05:03.023542 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.023517    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container" containerID="cri-o://ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d" gracePeriod=30
Oct 30 19:05:03.023637 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.023542    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kube-rbac-proxy" containerID="cri-o://00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432" gracePeriod=30
Oct 30 19:05:03.125332 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.125299    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:03.295348 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.295230    4938 generic.go:356] "Generic (PLEG): container finished" podID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerID="00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432" exitCode=0
Oct 30 19:05:03.295348 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:03.295310    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerDied","Data":"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"}
Oct 30 19:05:05.962169 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:05.962118    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.45:8643/healthz\": dial tcp 10.133.0.45:8643: connect: connection refused"
Oct 30 19:05:08.265610 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.265584    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:05:08.318371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.318280    4938 generic.go:356] "Generic (PLEG): container finished" podID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerID="ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d" exitCode=0
Oct 30 19:05:08.318522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.318394    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"
Oct 30 19:05:08.318522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.318399    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerDied","Data":"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"}
Oct 30 19:05:08.318522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.318460    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh" event={"ID":"80d064b5-befd-4324-b5d3-94dbfacd5d02","Type":"ContainerDied","Data":"efe26b68c87388dea00eaa368e1ba2343c7d7a53e4e65fe9a761b7ae4dc7d461"}
Oct 30 19:05:08.318522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.318489    4938 scope.go:122] "RemoveContainer" containerID="00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"
Oct 30 19:05:08.326761 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.326742    4938 scope.go:122] "RemoveContainer" containerID="ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"
Oct 30 19:05:08.336106 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.336085    4938 scope.go:122] "RemoveContainer" containerID="35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"
Oct 30 19:05:08.345707 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.345684    4938 scope.go:122] "RemoveContainer" containerID="00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"
Oct 30 19:05:08.346038 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:05:08.346009    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432\": container with ID starting with 00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432 not found: ID does not exist" containerID="00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"
Oct 30 19:05:08.346145 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.346053    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432"} err="failed to get container status \"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432\": rpc error: code = NotFound desc = could not find container \"00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432\": container with ID starting with 00f420de6412073de5074a144e70600dc9cd2409f55560f80dac9db92bfba432 not found: ID does not exist"
Oct 30 19:05:08.346145 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.346085    4938 scope.go:122] "RemoveContainer" containerID="ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"
Oct 30 19:05:08.346418 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:05:08.346401    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d\": container with ID starting with ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d not found: ID does not exist" containerID="ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"
Oct 30 19:05:08.346489 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.346428    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d"} err="failed to get container status \"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d\": rpc error: code = NotFound desc = could not find container \"ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d\": container with ID starting with ef7e88d931f74fcf0148fae3fada5a6fad8a4eacdfd7979084308b901f66893d not found: ID does not exist"
Oct 30 19:05:08.346489 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.346444    4938 scope.go:122] "RemoveContainer" containerID="35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"
Oct 30 19:05:08.346706 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:05:08.346683    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6\": container with ID starting with 35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6 not found: ID does not exist" containerID="35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"
Oct 30 19:05:08.346817 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.346712    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6"} err="failed to get container status \"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6\": rpc error: code = NotFound desc = could not find container \"35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6\": container with ID starting with 35254e4f59c8ea5bc43d805783f7b926caa5f264ddf36d4be5672118b91c0bf6 not found: ID does not exist"
Oct 30 19:05:08.362617 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362586    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\" (UniqueName: \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\") pod \"80d064b5-befd-4324-b5d3-94dbfacd5d02\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") "
Oct 30 19:05:08.362752 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362624    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\") pod \"80d064b5-befd-4324-b5d3-94dbfacd5d02\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") "
Oct 30 19:05:08.362752 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362682    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\") pod \"80d064b5-befd-4324-b5d3-94dbfacd5d02\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") "
Oct 30 19:05:08.362752 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362722    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\") pod \"80d064b5-befd-4324-b5d3-94dbfacd5d02\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") "
Oct 30 19:05:08.362859 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362765    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\") pod \"80d064b5-befd-4324-b5d3-94dbfacd5d02\" (UID: \"80d064b5-befd-4324-b5d3-94dbfacd5d02\") "
Oct 30 19:05:08.363025 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.362997    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location" pod "80d064b5-befd-4324-b5d3-94dbfacd5d02" (UID: "80d064b5-befd-4324-b5d3-94dbfacd5d02"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:05:08.363174 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.363152    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert" pod "80d064b5-befd-4324-b5d3-94dbfacd5d02" (UID: "80d064b5-befd-4324-b5d3-94dbfacd5d02"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:08.363238 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.363201    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config" pod "80d064b5-befd-4324-b5d3-94dbfacd5d02" (UID: "80d064b5-befd-4324-b5d3-94dbfacd5d02"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:05:08.364876 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.364858    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l" pod "80d064b5-befd-4324-b5d3-94dbfacd5d02" (UID: "80d064b5-befd-4324-b5d3-94dbfacd5d02"). InnerVolumeSpecName "kube-api-access-hvk6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:05:08.364948 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.364918    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls" pod "80d064b5-befd-4324-b5d3-94dbfacd5d02" (UID: "80d064b5-befd-4324-b5d3-94dbfacd5d02"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:05:08.463789 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.463746    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvk6l\" (UniqueName: \"kubernetes.io/projected/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-api-access-hvk6l\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:05:08.463789 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.463783    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/80d064b5-befd-4324-b5d3-94dbfacd5d02-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:05:08.463789 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.463793    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:05:08.463789 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.463802    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/80d064b5-befd-4324-b5d3-94dbfacd5d02-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:05:08.464066 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.463810    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/80d064b5-befd-4324-b5d3-94dbfacd5d02-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:05:08.666023 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.665979    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:05:08.677520 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:08.677487    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-predictor-b9c5956c9-kkwzh"]
Oct 30 19:05:10.392676 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:10.392640    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" path="/var/lib/kubelet/pods/80d064b5-befd-4324-b5d3-94dbfacd5d02/volumes"
Oct 30 19:05:11.652465 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:11.652422    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:32.549978 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:05:32.549939    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:05:41.797647 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:05:41.797610    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:02.549857 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:06:02.549819    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:06:11.776203 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:11.776160    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:32.549296 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:06:32.549193    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:06:43.743953 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:43.743921    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:06:43.744497 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:43.744171    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container" containerID="cri-o://5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6" gracePeriod=30
Oct 30 19:06:43.744497 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:43.744199    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kube-rbac-proxy" containerID="cri-o://b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e" gracePeriod=30
Oct 30 19:06:43.774043 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:43.774006    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:44.704844 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:44.704807    4938 generic.go:356] "Generic (PLEG): container finished" podID="19157767-3c2a-42e0-af2b-852a9d613136" containerID="b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e" exitCode=0
Oct 30 19:06:44.705056 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:44.704889    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerDied","Data":"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"}
Oct 30 19:06:44.958423 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:44.958315    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.44:8643/healthz\": dial tcp 10.133.0.44:8643: connect: connection refused"
Oct 30 19:06:49.102690 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.102662    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:06:49.150036 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.149935    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\" (UniqueName: \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\") pod \"19157767-3c2a-42e0-af2b-852a9d613136\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") "
Oct 30 19:06:49.150036 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.149982    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\") pod \"19157767-3c2a-42e0-af2b-852a9d613136\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") "
Oct 30 19:06:49.150036 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150036    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\") pod \"19157767-3c2a-42e0-af2b-852a9d613136\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") "
Oct 30 19:06:49.150350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150106    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\" (UniqueName: \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\") pod \"19157767-3c2a-42e0-af2b-852a9d613136\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") "
Oct 30 19:06:49.150350 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150141    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\") pod \"19157767-3c2a-42e0-af2b-852a9d613136\" (UID: \"19157767-3c2a-42e0-af2b-852a9d613136\") "
Oct 30 19:06:49.150556 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150441    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config" pod "19157767-3c2a-42e0-af2b-852a9d613136" (UID: "19157767-3c2a-42e0-af2b-852a9d613136"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:49.150691 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150488    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location" pod "19157767-3c2a-42e0-af2b-852a9d613136" (UID: "19157767-3c2a-42e0-af2b-852a9d613136"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:49.150691 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.150594    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert" pod "19157767-3c2a-42e0-af2b-852a9d613136" (UID: "19157767-3c2a-42e0-af2b-852a9d613136"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:49.152310 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.152286    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5" pod "19157767-3c2a-42e0-af2b-852a9d613136" (UID: "19157767-3c2a-42e0-af2b-852a9d613136"). InnerVolumeSpecName "kube-api-access-zxsf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:49.152413 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.152338    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls" pod "19157767-3c2a-42e0-af2b-852a9d613136" (UID: "19157767-3c2a-42e0-af2b-852a9d613136"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:06:49.251193 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.251155    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/19157767-3c2a-42e0-af2b-852a9d613136-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:06:49.251193 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.251191    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:06:49.251193 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.251201    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19157767-3c2a-42e0-af2b-852a9d613136-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:06:49.251468 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.251213    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxsf5\" (UniqueName: \"kubernetes.io/projected/19157767-3c2a-42e0-af2b-852a9d613136-kube-api-access-zxsf5\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:06:49.251468 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.251224    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/19157767-3c2a-42e0-af2b-852a9d613136-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:06:49.725575 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.725536    4938 generic.go:356] "Generic (PLEG): container finished" podID="19157767-3c2a-42e0-af2b-852a9d613136" containerID="5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6" exitCode=0
Oct 30 19:06:49.725785 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.725587    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerDied","Data":"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"}
Oct 30 19:06:49.725785 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.725599    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"
Oct 30 19:06:49.725785 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.725650    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5" event={"ID":"19157767-3c2a-42e0-af2b-852a9d613136","Type":"ContainerDied","Data":"bb5008abeb4bb3bfba222fe70041dada8e55e8237889cfa7a44973bcbc5ee708"}
Oct 30 19:06:49.725785 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.725689    4938 scope.go:122] "RemoveContainer" containerID="b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"
Oct 30 19:06:49.734625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.734605    4938 scope.go:122] "RemoveContainer" containerID="5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"
Oct 30 19:06:49.741997 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.741971    4938 scope.go:122] "RemoveContainer" containerID="6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"
Oct 30 19:06:49.749237 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.749216    4938 scope.go:122] "RemoveContainer" containerID="b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"
Oct 30 19:06:49.749549 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:06:49.749531    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e\": container with ID starting with b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e not found: ID does not exist" containerID="b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"
Oct 30 19:06:49.749602 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.749560    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e"} err="failed to get container status \"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e\": rpc error: code = NotFound desc = could not find container \"b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e\": container with ID starting with b9b6b5bd9653591a2ae6f3e9d798a39a619830987a3c32c0bd5c80409185e23e not found: ID does not exist"
Oct 30 19:06:49.749602 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.749581    4938 scope.go:122] "RemoveContainer" containerID="5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"
Oct 30 19:06:49.749851 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:06:49.749828    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6\": container with ID starting with 5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6 not found: ID does not exist" containerID="5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"
Oct 30 19:06:49.749888 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.749860    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6"} err="failed to get container status \"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6\": rpc error: code = NotFound desc = could not find container \"5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6\": container with ID starting with 5874664c534b88a596fc60e98b2917e172a6b20715168a291faf548a10d039c6 not found: ID does not exist"
Oct 30 19:06:49.749888 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.749876    4938 scope.go:122] "RemoveContainer" containerID="6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"
Oct 30 19:06:49.750091 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:06:49.750076    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a\": container with ID starting with 6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a not found: ID does not exist" containerID="6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"
Oct 30 19:06:49.750134 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.750099    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a"} err="failed to get container status \"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a\": rpc error: code = NotFound desc = could not find container \"6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a\": container with ID starting with 6ec863e3736c3dfea41dddd8f22227fda220ba29acd468d2ff806e66c3fa071a not found: ID does not exist"
Oct 30 19:06:49.799354 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.799317    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:06:49.804366 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:49.804340    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-42351c4657/isvc-canary-promote-v2-predictor-dc54999c4-spmt5"]
Oct 30 19:06:50.392557 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:50.392522    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19157767-3c2a-42e0-af2b-852a9d613136" path="/var/lib/kubelet/pods/19157767-3c2a-42e0-af2b-852a9d613136/volumes"
Oct 30 19:06:50.717506 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:50.717424    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:55.767270 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:06:55.767213    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:00.532157 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:00.532121    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:00.793206 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:00.793093    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.771240 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.771194    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.829553 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829517    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:07:01.829832 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829818    4938 cpu_manager.go:414] "containerMap: removing container" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kube-rbac-proxy"
Oct 30 19:07:01.829832 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829831    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kube-rbac-proxy"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829847    4938 cpu_manager.go:414] "containerMap: removing container" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="storage-initializer"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829852    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="storage-initializer"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829859    4938 cpu_manager.go:414] "containerMap: removing container" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kube-rbac-proxy"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829864    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kube-rbac-proxy"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829875    4938 cpu_manager.go:414] "containerMap: removing container" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="storage-initializer"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829880    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="storage-initializer"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829897    4938 cpu_manager.go:414] "containerMap: removing container" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829902    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829910    4938 cpu_manager.go:414] "containerMap: removing container" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829914    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829961    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829969    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kserve-container"
Oct 30 19:07:01.829970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829976    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="80d064b5-befd-4324-b5d3-94dbfacd5d02" containerName="kube-rbac-proxy"
Oct 30 19:07:01.830503 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.829983    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="19157767-3c2a-42e0-af2b-852a9d613136" containerName="kube-rbac-proxy"
Oct 30 19:07:01.834440 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.834417    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:01.864909 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.864871    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-a883951e49\"/\"message-dumper-raw-cipn-kube-rbac-proxy-sar-config\""
Oct 30 19:07:01.864909 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.864894    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-a883951e49\"/\"openshift-service-ca.crt\""
Oct 30 19:07:01.864909 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.864905    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-a883951e49\"/\"message-dumper-raw-cipn-predictor-serving-cert\""
Oct 30 19:07:01.865148 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.864930    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-a883951e49\"/\"kube-root-ca.crt\""
Oct 30 19:07:01.865148 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.864949    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-a883951e49\"/\"default-dockercfg-9pjcn\""
Oct 30 19:07:01.865796 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.865779    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:07:01.904223 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.904190    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:01.939469 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.939432    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:01.939469 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.939468    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52jnf\" (UniqueName: \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:01.939672 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:01.939498    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.040800 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.040712    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.040800 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.040749    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52jnf\" (UniqueName: \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.041032 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.040918    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.041427 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.041405    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.043304 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.043286    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.062903 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.062870    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52jnf\" (UniqueName: \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\") pod \"message-dumper-raw-cipn-predictor-778c8475d9-85t9d\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") " pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.148718 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.148677    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.448094 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.448058    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:07:02.453990 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:07:02.453957    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde14f42_0ae6_495e_a45f_c07442910326.slice/crio-a22f954dfba33db217c180a9f4f4b80f0c9887586f024c0aec7f2325f41fea36 WatchSource:0}: Error finding container a22f954dfba33db217c180a9f4f4b80f0c9887586f024c0aec7f2325f41fea36: Status 404 returned error can't find the container with id a22f954dfba33db217c180a9f4f4b80f0c9887586f024c0aec7f2325f41fea36
Oct 30 19:07:02.551373 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:07:02.551328    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:07:02.781559 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.781468    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerStarted","Data":"90522dca2821fa58dc6fb984ac532f48c313f96085d0afa77f6785037759a1f5"}
Oct 30 19:07:02.781559 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.781529    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerStarted","Data":"09e7ab67ff7b9f76e4bb060f6ef2f957d9ff1c014ebf4fa8a8a8c74186b16210"}
Oct 30 19:07:02.781559 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.781549    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerStarted","Data":"a22f954dfba33db217c180a9f4f4b80f0c9887586f024c0aec7f2325f41fea36"}
Oct 30 19:07:02.781994 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.781608    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:02.828941 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:02.828885    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" podStartSLOduration=1.828872262 podStartE2EDuration="1.828872262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:02.827739284 +0000 UTC m=+1784.587747748" watchObservedRunningTime="2026-10-30 19:07:02.828872262 +0000 UTC m=+1784.588880725"
Oct 30 19:07:03.786306 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:03.786273    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:03.788366 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:03.788344    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:10.799040 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:10.799009    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:07:11.871531 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:11.871499    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:12.021242 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.021207    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:12.026847 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.026820    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:07:12.030420 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.030400    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.047783 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.047752    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-a883951e49\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 19:07:12.047903 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.047761    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-a883951e49\"/\"isvc-logger-raw-cipn-kube-rbac-proxy-sar-config\""
Oct 30 19:07:12.047903 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.047869    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-a883951e49\"/\"seaweedfs-s3-creds\""
Oct 30 19:07:12.060830 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.060794    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-a883951e49\"/\"isvc-logger-raw-cipn-predictor-serving-cert\""
Oct 30 19:07:12.061999 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.061976    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:07:12.118320 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.118283    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.118492 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.118335    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.118492 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.118357    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.118492 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.118466    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.118613 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.118510    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6j5f\" (UniqueName: \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.218962 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.218877    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.218962 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.218942    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219132 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.218973    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219132 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.219027    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219132 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.219061    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6j5f\" (UniqueName: \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.219497    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219736 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.219715    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.219782 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.219736    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.221552 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.221527    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.240639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.240610    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6j5f\" (UniqueName: \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\") pod \"isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") " pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.350215 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.350171    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:12.681525 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.681499    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:07:12.683909 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:07:12.683880    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d35f86_bb02_4295_9f32_c2a656298112.slice/crio-792020ed2a8be70a085543dbcb2c034a0c157f33adf25324aef6ffa296ed3ae1 WatchSource:0}: Error finding container 792020ed2a8be70a085543dbcb2c034a0c157f33adf25324aef6ffa296ed3ae1: Status 404 returned error can't find the container with id 792020ed2a8be70a085543dbcb2c034a0c157f33adf25324aef6ffa296ed3ae1
Oct 30 19:07:12.823122 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.823075    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerStarted","Data":"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"}
Oct 30 19:07:12.823333 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:12.823153    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerStarted","Data":"792020ed2a8be70a085543dbcb2c034a0c157f33adf25324aef6ffa296ed3ae1"}
Oct 30 19:07:16.842541 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:16.842440    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d35f86-bb02-4295-9f32-c2a656298112" containerID="e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a" exitCode=0
Oct 30 19:07:16.842541 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:16.842518    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerDied","Data":"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"}
Oct 30 19:07:17.848907 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.848869    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerStarted","Data":"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"}
Oct 30 19:07:17.849341 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.848941    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerStarted","Data":"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"}
Oct 30 19:07:17.849341 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.848967    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerStarted","Data":"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"}
Oct 30 19:07:17.849341 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.849267    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:17.849496 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.849412    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:17.850543 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.850516    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:17.922163 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:17.921772    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podStartSLOduration=2.082603853 podStartE2EDuration="5.921751751s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.839147898s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:17.90829264 +0000 UTC m=+1799.668301101" watchObservedRunningTime="2026-10-30 19:07:17.921751751 +0000 UTC m=+1799.681760215"
Oct 30 19:07:18.551357 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.551331    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 19:07:18.555704 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.555679    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 19:07:18.567737 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.567709    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 19:07:18.577498 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.577467    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-k5ctz_861ecda3-10bb-451b-bf57-f62ef9cd4129/ovn-acl-logging/0.log"
Oct 30 19:07:18.852999 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.852951    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:18.852999 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.852992    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:18.854184 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:18.854155    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:19.857495 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:19.857450    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:19.857992 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:19.857972    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:24.866634 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:24.866604    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:07:24.867341 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:24.867306    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:24.867613 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:24.867591    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:32.550681 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:07:32.550641    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:07:32.845998 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:32.845958    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:33.002448 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:33.002410    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:34.841266 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:34.841209    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:34.867514 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:34.867467    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:34.867971 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:34.867942    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:39.357142 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:39.357105    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:40.377064 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:40.377011    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:40.520201 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:40.520164    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:44.867970 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:44.867924    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:44.868455 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:44.868438    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:07:54.867995 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:54.867940    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:07:54.868559 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:07:54.868446    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:02.550053 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:08:02.549964    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:08:04.867846 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:04.867805    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:08:04.868403 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:04.868368    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:12.050191 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:12.050153    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.867377 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:14.867327    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:08:14.867905 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:14.867830    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:24.868009 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:24.867953    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:08:24.868467 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:24.868428    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:32.549483 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:08:32.549447    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:08:34.868178 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:34.868130    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:08:34.868590 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:34.868565    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:08:44.868432 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:44.868396    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:08:44.868966 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:44.868623    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:08:57.363872 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:57.363841    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-a883951e49_message-dumper-raw-cipn-predictor-778c8475d9-85t9d_fde14f42-0ae6-495e-a45f-c07442910326/kserve-container/0.log"
Oct 30 19:08:57.978837 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:57.978787    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:08:57.979158 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:57.979065    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" containerID="cri-o://bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301" gracePeriod=30
Oct 30 19:08:57.979158 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:57.979104    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" containerID="cri-o://a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4" gracePeriod=30
Oct 30 19:08:57.979488 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:57.979451    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" containerID="cri-o://ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8" gracePeriod=30
Oct 30 19:08:58.070522 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.070483    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:58.105621 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.105587    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:08:58.105821 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.105801    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kserve-container" containerID="cri-o://09e7ab67ff7b9f76e4bb060f6ef2f957d9ff1c014ebf4fa8a8a8c74186b16210" gracePeriod=30
Oct 30 19:08:58.105916 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.105891    4938 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kube-rbac-proxy" containerID="cri-o://90522dca2821fa58dc6fb984ac532f48c313f96085d0afa77f6785037759a1f5" gracePeriod=30
Oct 30 19:08:58.220580 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.220544    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:58.254031 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.253929    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d35f86-bb02-4295-9f32-c2a656298112" containerID="ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8" exitCode=0
Oct 30 19:08:58.254031 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.254006    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerDied","Data":"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"}
Oct 30 19:08:58.256331 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.256300    4938 generic.go:356] "Generic (PLEG): container finished" podID="fde14f42-0ae6-495e-a45f-c07442910326" containerID="90522dca2821fa58dc6fb984ac532f48c313f96085d0afa77f6785037759a1f5" exitCode=0
Oct 30 19:08:58.256331 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.256326    4938 generic.go:356] "Generic (PLEG): container finished" podID="fde14f42-0ae6-495e-a45f-c07442910326" containerID="09e7ab67ff7b9f76e4bb060f6ef2f957d9ff1c014ebf4fa8a8a8c74186b16210" exitCode=2
Oct 30 19:08:58.256541 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.256379    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerDied","Data":"90522dca2821fa58dc6fb984ac532f48c313f96085d0afa77f6785037759a1f5"}
Oct 30 19:08:58.256541 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.256433    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerDied","Data":"09e7ab67ff7b9f76e4bb060f6ef2f957d9ff1c014ebf4fa8a8a8c74186b16210"}
Oct 30 19:08:58.356439 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.356413    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:08:58.477033 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.476995    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\") pod \"fde14f42-0ae6-495e-a45f-c07442910326\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") "
Oct 30 19:08:58.477677 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.477072    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\" (UniqueName: \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\") pod \"fde14f42-0ae6-495e-a45f-c07442910326\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") "
Oct 30 19:08:58.477677 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.477191    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\") pod \"fde14f42-0ae6-495e-a45f-c07442910326\" (UID: \"fde14f42-0ae6-495e-a45f-c07442910326\") "
Oct 30 19:08:58.477677 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.477456    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config" pod "fde14f42-0ae6-495e-a45f-c07442910326" (UID: "fde14f42-0ae6-495e-a45f-c07442910326"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:58.479496 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.479470    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls" pod "fde14f42-0ae6-495e-a45f-c07442910326" (UID: "fde14f42-0ae6-495e-a45f-c07442910326"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:08:58.479585 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.479473    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf" pod "fde14f42-0ae6-495e-a45f-c07442910326" (UID: "fde14f42-0ae6-495e-a45f-c07442910326"). InnerVolumeSpecName "kube-api-access-52jnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:58.578108 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.578074    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fde14f42-0ae6-495e-a45f-c07442910326-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:08:58.578108 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.578106    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fde14f42-0ae6-495e-a45f-c07442910326-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:08:58.578336 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:58.578117    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52jnf\" (UniqueName: \"kubernetes.io/projected/fde14f42-0ae6-495e-a45f-c07442910326-kube-api-access-52jnf\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:08:59.262131 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.262042    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d" event={"ID":"fde14f42-0ae6-495e-a45f-c07442910326","Type":"ContainerDied","Data":"a22f954dfba33db217c180a9f4f4b80f0c9887586f024c0aec7f2325f41fea36"}
Oct 30 19:08:59.262131 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.262065    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"
Oct 30 19:08:59.262131 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.262115    4938 scope.go:122] "RemoveContainer" containerID="90522dca2821fa58dc6fb984ac532f48c313f96085d0afa77f6785037759a1f5"
Oct 30 19:08:59.270670 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.270641    4938 scope.go:122] "RemoveContainer" containerID="09e7ab67ff7b9f76e4bb060f6ef2f957d9ff1c014ebf4fa8a8a8c74186b16210"
Oct 30 19:08:59.323621 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.323584    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:08:59.336768 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.336735    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-a883951e49/message-dumper-raw-cipn-predictor-778c8475d9-85t9d"]
Oct 30 19:08:59.858057 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:08:59.858003    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:00.392816 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:00.392777    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde14f42-0ae6-495e-a45f-c07442910326" path="/var/lib/kubelet/pods/fde14f42-0ae6-495e-a45f-c07442910326/volumes"
Oct 30 19:09:02.549712 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:02.549669    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:09:03.280621 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:03.280586    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d35f86-bb02-4295-9f32-c2a656298112" containerID="bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301" exitCode=0
Oct 30 19:09:03.280846 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:03.280662    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerDied","Data":"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"}
Oct 30 19:09:04.319638 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:04.319603    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:04.857856 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:04.857818    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:04.867358 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:04.867320    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:09:04.867639 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:04.867618    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:09:09.858029 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:09.857979    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:09.858463 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:09.858130    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:09:12.157041 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:12.157004    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:12.294003 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:12.293968    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:14.857879 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:14.857825    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:14.867816 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:14.867778    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:09:14.868145 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:14.868119    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:09:19.858535 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:19.858485    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:24.858579 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:24.858535    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.47:8643/healthz\": dial tcp 10.133.0.47:8643: connect: connection refused"
Oct 30 19:09:24.868039 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:24.867998    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.47:8080: connect: connection refused"
Oct 30 19:09:24.868225 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:24.868151    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:09:24.868585 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:24.868560    4938 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:09:24.868688 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:24.868675    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:09:28.128692 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.128664    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:09:28.186422 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186387    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\") pod \"38d35f86-bb02-4295-9f32-c2a656298112\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") "
Oct 30 19:09:28.186601 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186470    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\") pod \"38d35f86-bb02-4295-9f32-c2a656298112\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") "
Oct 30 19:09:28.186601 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186520    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\" (UniqueName: \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\") pod \"38d35f86-bb02-4295-9f32-c2a656298112\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") "
Oct 30 19:09:28.186601 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186567    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\") pod \"38d35f86-bb02-4295-9f32-c2a656298112\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") "
Oct 30 19:09:28.186752 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186613    4938 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\") pod \"38d35f86-bb02-4295-9f32-c2a656298112\" (UID: \"38d35f86-bb02-4295-9f32-c2a656298112\") "
Oct 30 19:09:28.186819 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186768    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location" pod "38d35f86-bb02-4295-9f32-c2a656298112" (UID: "38d35f86-bb02-4295-9f32-c2a656298112"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:09:28.187011 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.186984    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert" pod "38d35f86-bb02-4295-9f32-c2a656298112" (UID: "38d35f86-bb02-4295-9f32-c2a656298112"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:28.187092 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.187010    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config" pod "38d35f86-bb02-4295-9f32-c2a656298112" (UID: "38d35f86-bb02-4295-9f32-c2a656298112"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:09:28.188646 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.188610    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls" pod "38d35f86-bb02-4295-9f32-c2a656298112" (UID: "38d35f86-bb02-4295-9f32-c2a656298112"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:09:28.188750 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.188729    4938 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f" pod "38d35f86-bb02-4295-9f32-c2a656298112" (UID: "38d35f86-bb02-4295-9f32-c2a656298112"). InnerVolumeSpecName "kube-api-access-j6j5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:09:28.287625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.287533    4938 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6j5f\" (UniqueName: \"kubernetes.io/projected/38d35f86-bb02-4295-9f32-c2a656298112-kube-api-access-j6j5f\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:09:28.287625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.287567    4938 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-cabundle-cert\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:09:28.287625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.287579    4938 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/38d35f86-bb02-4295-9f32-c2a656298112-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:09:28.287625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.287588    4938 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/38d35f86-bb02-4295-9f32-c2a656298112-kserve-provision-location\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:09:28.287625 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.287597    4938 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/38d35f86-bb02-4295-9f32-c2a656298112-proxy-tls\") on node \"ip-10-0-132-168.ec2.internal\" DevicePath \"\""
Oct 30 19:09:28.384463 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.384430    4938 generic.go:356] "Generic (PLEG): container finished" podID="38d35f86-bb02-4295-9f32-c2a656298112" containerID="a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4" exitCode=0
Oct 30 19:09:28.384654 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.384494    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerDied","Data":"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"}
Oct 30 19:09:28.384654 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.384501    4938 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"
Oct 30 19:09:28.384654 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.384553    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz" event={"ID":"38d35f86-bb02-4295-9f32-c2a656298112","Type":"ContainerDied","Data":"792020ed2a8be70a085543dbcb2c034a0c157f33adf25324aef6ffa296ed3ae1"}
Oct 30 19:09:28.384654 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.384584    4938 scope.go:122] "RemoveContainer" containerID="a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"
Oct 30 19:09:28.393608 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.393586    4938 scope.go:122] "RemoveContainer" containerID="ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"
Oct 30 19:09:28.401548 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.401526    4938 scope.go:122] "RemoveContainer" containerID="bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"
Oct 30 19:09:28.408797 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.408776    4938 scope.go:122] "RemoveContainer" containerID="e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"
Oct 30 19:09:28.416151 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.416128    4938 scope.go:122] "RemoveContainer" containerID="a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"
Oct 30 19:09:28.416728 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:28.416703    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4\": container with ID starting with a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4 not found: ID does not exist" containerID="a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"
Oct 30 19:09:28.416802 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.416742    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4"} err="failed to get container status \"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4\": rpc error: code = NotFound desc = could not find container \"a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4\": container with ID starting with a06b7371dfa43b7096cb30d352e2015790e88f3fba705ed889bbec32106805e4 not found: ID does not exist"
Oct 30 19:09:28.416802 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.416766    4938 scope.go:122] "RemoveContainer" containerID="ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"
Oct 30 19:09:28.417009 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:28.416993    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8\": container with ID starting with ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8 not found: ID does not exist" containerID="ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"
Oct 30 19:09:28.417052 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.417013    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8"} err="failed to get container status \"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8\": rpc error: code = NotFound desc = could not find container \"ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8\": container with ID starting with ecd80d1b1fc6bde886b4b92bd9a79fdba5b6e5d995db92cab4c0eef75dd44bd8 not found: ID does not exist"
Oct 30 19:09:28.417052 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.417026    4938 scope.go:122] "RemoveContainer" containerID="bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"
Oct 30 19:09:28.417280 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:28.417233    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301\": container with ID starting with bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301 not found: ID does not exist" containerID="bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"
Oct 30 19:09:28.417402 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.417286    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301"} err="failed to get container status \"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301\": rpc error: code = NotFound desc = could not find container \"bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301\": container with ID starting with bd65a8e6f0bcce3325d611676d4ee7f42f5caf93af7f3bf0033f60f8b6d79301 not found: ID does not exist"
Oct 30 19:09:28.417402 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.417310    4938 scope.go:122] "RemoveContainer" containerID="e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"
Oct 30 19:09:28.417567 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:28.417552    4938 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a\": container with ID starting with e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a not found: ID does not exist" containerID="e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"
Oct 30 19:09:28.417609 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.417570    4938 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a"} err="failed to get container status \"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a\": rpc error: code = NotFound desc = could not find container \"e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a\": container with ID starting with e01bdbf5485fc28d45c76c1fe5ca4283a3e21c661ad4eea4439b17328aa78b9a not found: ID does not exist"
Oct 30 19:09:28.441371 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.441335    4938 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:09:28.445266 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:28.445218    4938 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-a883951e49/isvc-logger-raw-cipn-predictor-6c9b866f7d-mqblz"]
Oct 30 19:09:29.366547 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:29.366511    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:30.391758 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:30.391720    4938 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38d35f86-bb02-4295-9f32-c2a656298112" path="/var/lib/kubelet/pods/38d35f86-bb02-4295-9f32-c2a656298112/volumes"
Oct 30 19:09:31.239651 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:31.239610    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:31.396674 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:31.396623    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:32.549159 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:09:32.549121    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:09:35.752026 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:35.751990    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:35.897296 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:09:35.897235    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:02.549020 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:10:02.548978    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:10:12.299392 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:10:12.299351    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:32.549358 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:10:32.549318    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:11:02.549293 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:11:02.549195    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:11:12.408772 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:12.408737    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.549408 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:11:32.549371    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:11:37.989804 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:37.989769    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.268314 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268198    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mzl4/must-gather-wq2qm"]
Oct 30 19:11:38.268620 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268602    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268621    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268646    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268654    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268674    4938 cpu_manager.go:414] "containerMap: removing container" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kube-rbac-proxy"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268682    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kube-rbac-proxy"
Oct 30 19:11:38.268706 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268707    4938 cpu_manager.go:414] "containerMap: removing container" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268714    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268729    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="storage-initializer"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268736    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="storage-initializer"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268746    4938 cpu_manager.go:414] "containerMap: removing container" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268753    4938 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268834    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268850    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kserve-container"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268860    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="kube-rbac-proxy"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268870    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="38d35f86-bb02-4295-9f32-c2a656298112" containerName="agent"
Oct 30 19:11:38.269018 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.268880    4938 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde14f42-0ae6-495e-a45f-c07442910326" containerName="kube-rbac-proxy"
Oct 30 19:11:38.272027 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.272005    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.289508 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.289483    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mzl4\"/\"kube-root-ca.crt\""
Oct 30 19:11:38.302471 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.302449    4938 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8mzl4\"/\"default-dockercfg-9xkl2\""
Oct 30 19:11:38.302471 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.302452    4938 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mzl4\"/\"openshift-service-ca.crt\""
Oct 30 19:11:38.303417 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.303394    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mzl4/must-gather-wq2qm"]
Oct 30 19:11:38.384335 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.384302    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-must-gather-output\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.384499 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.384355    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7fxb\" (UniqueName: \"kubernetes.io/projected/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-kube-api-access-h7fxb\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.485146 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.485112    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-must-gather-output\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.485344 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.485160    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7fxb\" (UniqueName: \"kubernetes.io/projected/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-kube-api-access-h7fxb\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.485517 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.485493    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-must-gather-output\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.505544 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.505520    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7fxb\" (UniqueName: \"kubernetes.io/projected/58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1-kube-api-access-h7fxb\") pod \"must-gather-wq2qm\" (UID: \"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1\") " pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.595642 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.595614    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mzl4/must-gather-wq2qm"
Oct 30 19:11:38.889084 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.889003    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mzl4/must-gather-wq2qm"]
Oct 30 19:11:38.891960 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:11:38.891933    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58f92b1e_3ffa_495c_ab0f_ff5fc2872ae1.slice/crio-5bd07c33eacf35847dde7a5aa0318d5fe2844b63f7bd88a218d07e69328e6641 WatchSource:0}: Error finding container 5bd07c33eacf35847dde7a5aa0318d5fe2844b63f7bd88a218d07e69328e6641: Status 404 returned error can't find the container with id 5bd07c33eacf35847dde7a5aa0318d5fe2844b63f7bd88a218d07e69328e6641
Oct 30 19:11:38.893787 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:38.893766    4938 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:11:39.205705 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:39.205615    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:39.874388 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:39.874345    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mzl4/must-gather-wq2qm" event={"ID":"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1","Type":"ContainerStarted","Data":"71bdee1fea5ce9120690aba913aba14636fe7ea67eb078addd47d48086e5b22a"}
Oct 30 19:11:39.874510 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:39.874399    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mzl4/must-gather-wq2qm" event={"ID":"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1","Type":"ContainerStarted","Data":"5bd07c33eacf35847dde7a5aa0318d5fe2844b63f7bd88a218d07e69328e6641"}
Oct 30 19:11:40.880171 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:40.880134    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mzl4/must-gather-wq2qm" event={"ID":"58f92b1e-3ffa-495c-ab0f-ff5fc2872ae1","Type":"ContainerStarted","Data":"8afe76a9245740d507d2ec0bf365f97e500ea7ca1fccca6134e89d4d83cd3389"}
Oct 30 19:11:40.925485 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:40.925434    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-8mzl4/must-gather-wq2qm" podStartSLOduration=2.0522878 podStartE2EDuration="2.925421158s" totalImagesPullingTime="873.133358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:40.924581759 +0000 UTC m=+2062.684590224" watchObservedRunningTime="2026-10-30 19:11:40.925421158 +0000 UTC m=+2062.685429612"
Oct 30 19:11:42.543196 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:42.543169    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kp5w5_0e6538b1-d3ff-4e67-81e0-e147ce5d03db/global-pull-secret-syncer/0.log"
Oct 30 19:11:43.091974 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.091947    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-5kbfh_e69d8d91-a4a3-412d-ab1b-d9a8409580fa/connection-checker/0.log"
Oct 30 19:11:43.209333 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.209300    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-96hgm_4d127029-258f-4c97-a9b8-d2cf85123890/connection-checker/0.log"
Oct 30 19:11:43.324895 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.324867    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-zh56w_7265c58c-0d6e-4fe4-98af-9db6238dd3b8/connection-checker/0.log"
Oct 30 19:11:43.455409 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.455322    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log"
Oct 30 19:11:43.688337 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.688306    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:43.830402 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.830373    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-168.ec2.internal_d35ca846eedff7c0a375d6f9a1bb598c/haproxy/0.log"
Oct 30 19:11:43.833428 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:43.833403    4938 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6bz4h_fca72a21-1b7a-4005-9baa-970f86ebd652/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:49.752207 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:49.752176    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-csnrg_13252367-d85f-4a6a-9eae-2e22cfbb5c4f/cluster-monitoring-operator/0.log"
Oct 30 19:11:51.190154 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.190121    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfjt5_cd97b741-9720-4db9-811d-0fc9ef8fef22/node-exporter/0.log"
Oct 30 19:11:51.304518 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.304490    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfjt5_cd97b741-9720-4db9-811d-0fc9ef8fef22/kube-rbac-proxy/0.log"
Oct 30 19:11:51.438895 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.438870    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tfjt5_cd97b741-9720-4db9-811d-0fc9ef8fef22/init-textfile/0.log"
Oct 30 19:11:51.470963 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.470883    4938 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"]
Oct 30 19:11:51.474326 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.474301    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.505081 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.505042    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"]
Oct 30 19:11:51.549880 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.549835    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-podres\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.550041 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.549893    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-proc\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.550041 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.549969    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhb5\" (UniqueName: \"kubernetes.io/projected/34ca9559-4313-43fd-ba1a-0f5d36b34e05-kube-api-access-chhb5\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.550041 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.550000    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-lib-modules\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.550146 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.550105    4938 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-sys\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650561 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650521    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-sys\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650720 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650633    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-sys\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650720 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650664    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-podres\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650720 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650712    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-proc\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650842 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650750    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chhb5\" (UniqueName: \"kubernetes.io/projected/34ca9559-4313-43fd-ba1a-0f5d36b34e05-kube-api-access-chhb5\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650842 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650760    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-podres\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650842 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650777    4938 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-lib-modules\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650842 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650813    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-proc\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.650975 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.650900    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34ca9559-4313-43fd-ba1a-0f5d36b34e05-lib-modules\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.670768 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.670741    4938 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhb5\" (UniqueName: \"kubernetes.io/projected/34ca9559-4313-43fd-ba1a-0f5d36b34e05-kube-api-access-chhb5\") pod \"perf-node-gather-daemonset-j5hw4\" (UID: \"34ca9559-4313-43fd-ba1a-0f5d36b34e05\") " pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:51.787238 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:51.787155    4938 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:52.091722 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:52.091690    4938 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"]
Oct 30 19:11:52.094596 ip-10-0-132-168 kubenswrapper[4938]: W1030 19:11:52.094564    4938 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34ca9559_4313_43fd_ba1a_0f5d36b34e05.slice/crio-1fc97c21ec207be41bfd0ecb36612e90e12e26c93daf2b0e488e944fc93fda8b WatchSource:0}: Error finding container 1fc97c21ec207be41bfd0ecb36612e90e12e26c93daf2b0e488e944fc93fda8b: Status 404 returned error can't find the container with id 1fc97c21ec207be41bfd0ecb36612e90e12e26c93daf2b0e488e944fc93fda8b
Oct 30 19:11:52.955760 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:52.955723    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4" event={"ID":"34ca9559-4313-43fd-ba1a-0f5d36b34e05","Type":"ContainerStarted","Data":"eb73a9d9950d1e161c7bc46e0a30e1124efd4bd3ec9a8fa9f77f01e7f9ec2e26"}
Oct 30 19:11:52.956215 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:52.955777    4938 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4" event={"ID":"34ca9559-4313-43fd-ba1a-0f5d36b34e05","Type":"ContainerStarted","Data":"1fc97c21ec207be41bfd0ecb36612e90e12e26c93daf2b0e488e944fc93fda8b"}
Oct 30 19:11:52.956215 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:52.955850    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:11:52.997861 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:52.997808    4938 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4" podStartSLOduration=1.99779687 podStartE2EDuration="1.99779687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:11:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:52.997121681 +0000 UTC m=+2074.757130144" watchObservedRunningTime="2026-10-30 19:11:52.99779687 +0000 UTC m=+2074.757805329"
Oct 30 19:11:55.404358 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:55.404322    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-r6r8p_f35ce557-816c-4b2e-97cc-19937efb59db/networking-console-plugin/0.log"
Oct 30 19:11:56.301037 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:56.301006    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/2.log"
Oct 30 19:11:56.306515 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:56.306498    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-rzcch_808b5fa2-8623-4639-870a-be16d38cafeb/console-operator/3.log"
Oct 30 19:11:58.219874 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:58.219846    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-q4xsh_310d68ba-e235-4caf-ba78-c1940e74354c/volume-data-source-validator/0.log"
Oct 30 19:11:58.970459 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:11:58.970429    4938 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8mzl4/perf-node-gather-daemonset-j5hw4"
Oct 30 19:12:00.265788 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:00.265757    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6w2g6_1d135ba2-ec2b-4cc2-a1a9-478ba254e16f/dns/0.log"
Oct 30 19:12:00.377329 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:00.377294    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6w2g6_1d135ba2-ec2b-4cc2-a1a9-478ba254e16f/kube-rbac-proxy/0.log"
Oct 30 19:12:00.848132 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:00.848105    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j4266_7ba4b2b7-3196-4769-aaa9-8c40dd282893/dns-node-resolver/0.log"
Oct 30 19:12:02.097849 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:02.097817    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ghjt5_279b261c-91e6-427e-9e19-3f1a42e60e76/node-ca/0.log"
Oct 30 19:12:02.558336 ip-10-0-132-168 kubenswrapper[4938]: E1030 19:12:02.558260    4938 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-p9vkt" containerName="extractor"
Oct 30 19:12:03.946494 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:03.946465    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c6947b789-btt56_666afb05-0236-4db8-8229-0871af2ff16f/router/0.log"
Oct 30 19:12:05.063856 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:05.063827    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tkz7q_cff1eb5a-0fc0-47ad-adff-12489a13e317/serve-healthcheck-canary/0.log"
Oct 30 19:12:05.883940 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:05.883905    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-mvhgb_bf0111a4-adeb-451d-a0c2-17469dce5b90/insights-operator/0.log"
Oct 30 19:12:05.884381 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:05.884368    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-mvhgb_bf0111a4-adeb-451d-a0c2-17469dce5b90/insights-operator/1.log"
Oct 30 19:12:06.368345 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:06.368318    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9vkt_8feb3890-bb59-4850-a7cf-7c53c9048ad1/kube-rbac-proxy/0.log"
Oct 30 19:12:06.481564 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:06.481537    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9vkt_8feb3890-bb59-4850-a7cf-7c53c9048ad1/exporter/0.log"
Oct 30 19:12:06.599911 ip-10-0-132-168 kubenswrapper[4938]: I1030 19:12:06.599887    4938 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-p9vkt_8feb3890-bb59-4850-a7cf-7c53c9048ad1/extractor/0.log"