Jul 13 21:11:08.451382 ip-10-0-143-242 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.952423    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956406    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956422    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956426    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956431    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956436    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956439    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956443    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:08.976581 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956446    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956449    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956452    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956455    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956458    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956462    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956464    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956467    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956470    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956472    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956476    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956489    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956493    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956495    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956499    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956502    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956506    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956509    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956512    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956515    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:08.977955 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956518    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956520    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956523    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956526    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956529    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956531    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956535    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956538    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956541    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956543    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956546    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956549    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956551    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956554    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956556    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956559    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956562    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956565    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956568    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956570    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:08.980326 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956573    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956576    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956579    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956582    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956585    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956588    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956591    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956593    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956596    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956599    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956602    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956606    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956610    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956612    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956615    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956618    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956621    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956624    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956627    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956630    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:08.981228 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956632    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956636    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956640    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956643    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956647    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956650    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956652    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956655    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956657    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956660    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956663    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956667    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956670    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956673    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956676    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956678    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956681    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956684    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956686    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:08.982543 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956690    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956692    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956697    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956701    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956704    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956707    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956710    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956713    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956716    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956718    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956721    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956724    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.956726    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957521    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957529    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957535    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957539    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957543    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957548    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:08.983422 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957552    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957555    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957559    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957562    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957566    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957569    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957572    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957574    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957577    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957580    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957583    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957586    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957589    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957592    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957595    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957598    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957600    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957603    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957606    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:08.984736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957609    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957612    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957615    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957618    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957622    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957626    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957629    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957632    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957635    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957638    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957641    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957643    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957646    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957652    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957655    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957658    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957660    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957663    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957666    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957669    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:08.986965 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957671    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957674    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957677    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957681    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957684    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957687    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957689    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957692    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957695    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957697    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957700    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957703    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957706    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957709    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957712    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957715    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957718    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957720    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957723    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957726    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:08.987665 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957728    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957731    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957734    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957737    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957739    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957744    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957748    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957753    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957756    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957759    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957762    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957764    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957767    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957769    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957772    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957775    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957778    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957781    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957783    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:08.988341 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957786    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957788    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957791    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957794    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957797    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957799    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957802    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957805    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957808    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957810    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957813    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957816    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957818    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957821    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.957824    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958334    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958341    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958345    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958350    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958353    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:08.989017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958358    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958361    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958364    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958367    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958370    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958374    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958377    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958380    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958383    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958386    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958388    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958391    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958394    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958396    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958399    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958402    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958404    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958407    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958410    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:08.989669 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958413    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958415    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958418    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958421    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958423    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958426    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958429    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958431    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958434    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958437    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958439    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958444    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958447    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958452    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958455    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958457    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958460    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958463    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958466    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958468    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:08.990343 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958471    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958474    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958477    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958479    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958482    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958485    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958488    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958490    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958493    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958496    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958499    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958502    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958504    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958507    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958509    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958512    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958515    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958518    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958520    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958523    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:08.991017 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958525    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958528    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958530    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958535    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958537    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958542    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958545    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958548    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958551    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958554    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958556    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958559    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958562    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958564    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958567    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958571    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958575    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958578    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958581    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:08.991650 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958584    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958587    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958591    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958594    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958597    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958599    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958603    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958605    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958608    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958610    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958613    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958615    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958620    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958623    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958626    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.958629    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958712    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958721    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958729    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958734    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 21:11:08.992356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958739    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958742    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958747    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958753    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958756    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958759    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958763    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958766    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958770    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958773    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958776    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958780    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958783    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958787    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958791    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958794    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958798    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958802    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958805    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958809    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958813    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958816    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958819    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958823    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 21:11:08.993041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958826    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958865    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958870    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958874    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958883    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958886    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958891    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958894    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958897    2577 flags.go:64] FLAG: --enable-server="true"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958902    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958906    2577 flags.go:64] FLAG: --event-burst="100"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958910    2577 flags.go:64] FLAG: --event-qps="50"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958913    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958916    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958919    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958924    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958927    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958930    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958933    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958937    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958940    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958943    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958946    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958949    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958953    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 21:11:08.993832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958956    2577 flags.go:64] FLAG: --feature-gates=""
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958960    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958963    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958966    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958970    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958973    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958977    2577 flags.go:64] FLAG: --help="false"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958980    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958983    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958986    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958989    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958995    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.958999    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959003    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959006    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959010    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959013    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959017    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959020    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959023    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959027    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959030    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959033    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959036    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 21:11:08.994672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959040    2577 flags.go:64] FLAG: --lock-file=""
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959043    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959046    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959049    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959055    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959058    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959062    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959065    2577 flags.go:64] FLAG: --logging-format="text"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959068    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959072    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959075    2577 flags.go:64] FLAG: --manifest-url=""
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959078    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959085    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959090    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959095    2577 flags.go:64] FLAG: --max-pods="110"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959099    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959102    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959105    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959108    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959112    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959116    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959120    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959128    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959132    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959135    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 21:11:08.995508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959138    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959143    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959150    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959153    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959157    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959160    2577 flags.go:64] FLAG: --port="10250"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959163    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959166    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09e906d76d3e22d77"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959170    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959173    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959176    2577 flags.go:64] FLAG: --register-node="true"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959179    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959183    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959186    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959189    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959192    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959195    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959199    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959202    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959205    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959208    2577 flags.go:64] FLAG: --runonce="false"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959211    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959215    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959218    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959221    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 21:11:08.996359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959224    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959229    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959233    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959236    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959241    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959244    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959247    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959250    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959254    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959258    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959261    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959266    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959270    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959273    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959278    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959281    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959284    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959288    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959291    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959294    2577 flags.go:64] FLAG: --v="2"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959299    2577 flags.go:64] FLAG: --version="false"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959304    2577 flags.go:64] FLAG: --vmodule=""
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959308    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959311    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 21:11:08.997249 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959435    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959441    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959444    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959447    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959450    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959452    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959455    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959458    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959461    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959465    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959468    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959471    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959475    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959478    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959481    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959484    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959487    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959490    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959494    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959497    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:08.998088 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959500    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959502    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959505    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959508    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959510    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959513    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959516    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959519    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959521    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959524    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959527    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959529    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959532    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959535    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959537    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959540    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959543    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959546    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959548    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959551    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:08.998752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959554    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959558    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959560    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959563    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959567    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959570    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959573    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959576    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959579    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959583    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959587    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959591    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959594    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959598    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959601    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959604    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959607    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959610    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959613    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:08.999432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959616    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959619    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959621    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959625    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959627    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959630    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959633    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959635    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959638    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959641    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959645    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959649    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959652    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959655    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959658    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959661    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959664    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959668    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959670    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:09.000079 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959673    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959676    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959679    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959682    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959685    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959687    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959691    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959693    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959696    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959699    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959701    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959704    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959707    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959710    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959712    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959715    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959717    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959720    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959723    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959725    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:09.000722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.959728    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.959734    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.966890    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.966910    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.966930    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.966940    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967001    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967008    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967011    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967015    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967018    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967022    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967025    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967028    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967031    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:09.001411 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967033    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967036    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967039    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967044    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967048    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967051    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967054    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967057    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967060    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967064    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967067    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967071    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967074    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967076    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967080    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967084    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967086    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967090    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967094    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:09.001939 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967096    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967100    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967103    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967106    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967111    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967114    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967118    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967121    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967124    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967126    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967129    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967132    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967136    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967138    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967141    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967144    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967147    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967149    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967152    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:09.002557 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967155    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967158    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967161    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967163    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967166    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967169    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967172    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967174    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967178    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967181    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967184    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967187    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967190    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967192    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967195    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967199    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967203    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967205    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967208    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967211    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:09.003208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967214    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967217    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967220    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967223    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967225    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967228    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967231    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967233    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967236    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967239    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967242    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967245    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967247    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967250    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967253    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967255    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967258    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967261    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967264    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967266    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:09.003897 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967270    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967273    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967276    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967281    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967285    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967288    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967290    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967293    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967297    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967300    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967302    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967305    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.967310    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967458    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967464    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:09.004609 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967468    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967471    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967474    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967477    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967480    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967483    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967486    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967489    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967493    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967497    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967502    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967505    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967508    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967524    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967528    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967531    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967534    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967537    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967542    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:09.005126 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967545    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967549    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967552    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967555    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967557    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967560    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967563    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967566    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967569    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967572    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967574    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967577    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967580    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967582    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967585    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967588    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967591    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967594    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967597    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967599    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:09.005752 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967602    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967605    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967607    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967610    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967613    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967616    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967618    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967621    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967625    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967629    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967632    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967638    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967641    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967645    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967648    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967651    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967654    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967657    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:09.006432 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967660    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967663    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967667    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967669    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967672    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967675    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967677    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967680    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967683    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967685    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967688    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967691    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967694    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967696    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967699    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967702    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967704    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967708    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967712    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967715    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:09.007048 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967718    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967721    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967723    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967726    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967729    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967733    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967736    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967740    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967743    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967746    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967749    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967752    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967754    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967757    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967760    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967763    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967766    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967769    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967772    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:09.007701 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:08.967775    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.967779    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.967934    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.972462    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.975012    2577 server.go:1019] "Starting client certificate rotation"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.975126    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.975169    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.978416    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.979665    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:08.999420    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.001957    2577 log.go:25] "Validated CRI v1 image API"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.002897    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 21:11:09.008354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.003391    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:11:09.008765 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.005983    2577 fs.go:135] Filesystem UUIDs: map[01ad9fee-d270-490a-a9c3-2679a728a2da:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fb07acb1-ef86-43d4-9f1d-1bcf0e93cdb2:/dev/nvme0n1p4]
Jul 13 21:11:09.008765 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.005999    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 21:11:09.016805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.016653    2577 manager.go:216] Machine: {Timestamp:2026-07-13 21:11:09.011445757 +0000 UTC m=+0.428372095 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098178 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28912c447f1ef429421e3d87e9061b SystemUUID:ec28912c-447f-1ef4-2942-1e3d87e9061b BootID:5e1baac9-b9e2-48ff-8c17-721411d6354f Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6b:6e:ad:65:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6b:6e:ad:65:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:e9:d1:ea:4a:46 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 21:11:09.016805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.016795    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 21:11:09.016952 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.016913    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 21:11:09.018154 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.018119    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 21:11:09.018322 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.018158    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-242.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 21:11:09.019394 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.019381    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 21:11:09.019437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.019397    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 21:11:09.019437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.019405    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:11:09.020397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.020382    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:11:09.021939 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.021925    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 21:11:09.025373 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025356    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:11:09.025513 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025503    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 21:11:09.025551 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025544    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 21:11:09.025587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025559    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 21:11:09.025587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025572    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 21:11:09.025587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025583    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 13 21:11:09.025703 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.025593    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 21:11:09.026592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.026118    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:11:09.026592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.026155    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:11:09.031397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.031371    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 21:11:09.032141 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.032117    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n9l6q"
Jul 13 21:11:09.033306 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033284    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 21:11:09.033388 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033337    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 21:11:09.033592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033580    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 21:11:09.033636 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033597    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 21:11:09.033636 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033604    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 21:11:09.033636 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033610    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 21:11:09.033636 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033619    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 21:11:09.033636 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.033624    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 21:11:09.034131 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034120    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 21:11:09.034131 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034130    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 21:11:09.034198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034138    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 21:11:09.034198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034144    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 21:11:09.034198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034154    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 21:11:09.034198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.034162    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 21:11:09.035024 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.035013    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 21:11:09.035024 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.035023    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 21:11:09.037858 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.037836    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-242.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 21:11:09.037963 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.037924    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-n9l6q"
Jul 13 21:11:09.038351 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.038330    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-242.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 21:11:09.038403 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.038333    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 21:11:09.039136 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.039105    2577 server.go:1268] "Started kubelet"
Jul 13 21:11:09.039420 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.039347    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 21:11:09.039506 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.039441    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 21:11:09.039559 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.039395    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 21:11:09.040220 ip-10-0-143-242 systemd[1]: Started Kubernetes Kubelet.
Jul 13 21:11:09.044570 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.044546    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 21:11:09.044853 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.044828    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 21:11:09.046683 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.046670    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 21:11:09.048705 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.048680    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 21:11:09.048786 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.048760    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 21:11:09.048921 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.048901    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.049528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049305    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 21:11:09.049528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049324    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 21:11:09.049528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049411    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 21:11:09.049528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049488    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 21:11:09.049528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049493    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049902    2577 factory.go:55] Registering systemd factory
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.049925    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.050432    2577 factory.go:153] Registering CRI-O factory
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.050450    2577 factory.go:223] Registration of the crio container factory successfully
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.050515    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.050540    2577 factory.go:103] Registering Raw factory
Jul 13 21:11:09.051810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.050559    2577 manager.go:1196] Started watching for new ooms in manager
Jul 13 21:11:09.052116 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.051937    2577 manager.go:349] Starting recovery of all containers
Jul 13 21:11:09.056327 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.056297    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:09.060674 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.060482    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-242.ec2.internal\" not found" node="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.063629 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.063597    2577 manager.go:354] Recovery completed
Jul 13 21:11:09.070490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.070471    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.072887 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.072864    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.072992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.072908    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.072992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.072925    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.073550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.073531    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 21:11:09.073550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.073538    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.073550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.073548    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 21:11:09.073724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.073570    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:11:09.074371 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.074354    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.074438 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.074419    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.074438 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.074432    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.075155 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.075143    2577 policy_none.go:49] "None policy: Start"
Jul 13 21:11:09.075200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.075165    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 21:11:09.075200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.075176    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 21:11:09.076277 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.076265    2577 policy_none.go:47] "Start"
Jul 13 21:11:09.111507 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.111489    2577 manager.go:337] "Starting Device Plugin manager"
Jul 13 21:11:09.111626 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.111531    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 21:11:09.111626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.111546    2577 server.go:87] "Starting device plugin registration server"
Jul 13 21:11:09.111889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.111874    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 21:11:09.111944 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.111891    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 21:11:09.112017 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.111999    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 21:11:09.112144 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.112109    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 21:11:09.112144 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.112127    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 21:11:09.112689 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.112666    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 21:11:09.112760 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.112729    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.174936 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.174893    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 21:11:09.200744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.176415    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 21:11:09.200744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.176465    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 21:11:09.200744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.176497    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 21:11:09.200744 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.176537    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 21:11:09.200744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.179045    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:09.212872 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.212845    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.214115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.214095    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.214218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.214128    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.214218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.214141    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.214218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.214171    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.224514 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.224487    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.224514 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.224516    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-242.ec2.internal\": node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.238660 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.238582    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.277931 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.277890    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"]
Jul 13 21:11:09.278057 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.277985    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.280014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.279994    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.280089 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.280035    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.280089 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.280050    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.281625 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.281611    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.281802 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.281766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.281864 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.281832    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.282495 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282476    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.282586 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282506    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.282586 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282517    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.282586 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282480    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.282698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282589    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.282698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.282600    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.283742 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.283723    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.283842 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.283752    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:09.284511 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.284492    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:09.284620 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.284531    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:09.284620 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.284548    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:09.310869 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.310835    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-242.ec2.internal\" not found" node="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.315811 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.315791    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-242.ec2.internal\" not found" node="ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.338824 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.338787    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.352252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.352226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.352350 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.352257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.352350 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.352276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ac4ccc85f99651906e4767386806ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-242.ec2.internal\" (UID: \"0ac4ccc85f99651906e4767386806ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.442034 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.442001    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.453151 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.453235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.453235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ac4ccc85f99651906e4767386806ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-242.ec2.internal\" (UID: \"0ac4ccc85f99651906e4767386806ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.453340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.453340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453296    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ac4ccc85f99651906e4767386806ad9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-242.ec2.internal\" (UID: \"0ac4ccc85f99651906e4767386806ad9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.453340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.453330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/329fa49187de2fbb90a0ad7755dac966-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal\" (UID: \"329fa49187de2fbb90a0ad7755dac966\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.542297 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.542214    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.617317 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.617278    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.618453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.618436    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:09.643067 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.643037    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.748398 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.748354    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.849681 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.849586    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.954703 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:09.954657    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:09.973398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.973371    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 21:11:09.973589 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.973563    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:11:09.973639 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:09.973584    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:11:10.042992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.042946    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:06:09 +0000 UTC" deadline="2028-03-31 23:21:08.518639066 +0000 UTC"
Jul 13 21:11:10.042992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.042984    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15050h9m58.475660714s"
Jul 13 21:11:10.055276 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:10.055244    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:10.155814 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:10.155706    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:10.256260 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:10.256213    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:10.290824 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.290767    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:10.357058 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:10.357024    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:10.438928 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:10.438888    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod329fa49187de2fbb90a0ad7755dac966.slice/crio-47daf8ec3c75fc6ee0efcca8da0d51d98b58147a91869de74d66fbcc63622883 WatchSource:0}: Error finding container 47daf8ec3c75fc6ee0efcca8da0d51d98b58147a91869de74d66fbcc63622883: Status 404 returned error can't find the container with id 47daf8ec3c75fc6ee0efcca8da0d51d98b58147a91869de74d66fbcc63622883
Jul 13 21:11:10.439224 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:10.439194    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac4ccc85f99651906e4767386806ad9.slice/crio-42cccbc71d4325c22db31bc1403d974109f03ca5f7449fe9f0ca351a6a8b256a WatchSource:0}: Error finding container 42cccbc71d4325c22db31bc1403d974109f03ca5f7449fe9f0ca351a6a8b256a: Status 404 returned error can't find the container with id 42cccbc71d4325c22db31bc1403d974109f03ca5f7449fe9f0ca351a6a8b256a
Jul 13 21:11:10.443905 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.443883    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:11:10.457665 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:10.457639    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-242.ec2.internal\" not found"
Jul 13 21:11:10.493104 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.493071    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:10.550329 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.550288    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:10.562573 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.562541    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:11:10.564624 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.564608    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal"
Jul 13 21:11:10.573965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.573928    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:11:10.894725 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.894628    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:10.928297 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:10.928265    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:11.027725 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.027687    2577 apiserver.go:52] "Watching apiserver"
Jul 13 21:11:11.034749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.034718    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 21:11:11.040366 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.040327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-rzds7","openshift-network-diagnostics/network-check-target-27v2r","kube-system/global-pull-secret-syncer-bjjbg","kube-system/konnectivity-agent-26wqg","kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gs2rp","openshift-multus/multus-7zgs6","openshift-network-operator/iptables-alerter-p5t77","openshift-ovn-kubernetes/ovnkube-node-v6skj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv","openshift-dns/node-resolver-zngbn","openshift-image-registry/node-ca-98x85","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal","openshift-multus/multus-additional-cni-plugins-9msh7"]
Jul 13 21:11:11.042398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.042366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.043968 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.043942    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045368    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045395    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-knsbn\""
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045417    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045436    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.045379    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.045626 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.045497    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:11.046248 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.046157    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 21:11:11.046248 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.046223    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vhcm9\""
Jul 13 21:11:11.046974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.046583    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.046974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.046658    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.046974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.046787    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.048376 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.048247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.048376 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.048335    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:11.049141 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.048984    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zzfvv\""
Jul 13 21:11:11.049277 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.049201    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 21:11:11.049347 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.049297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 21:11:11.049347 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.049204    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 21:11:11.050478 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.050388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.052467 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.052444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.053107 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.053087    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.053107 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.053101    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.055550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kddds\""
Jul 13 21:11:11.055550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055223    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 21:11:11.055550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055372    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pkztb\""
Jul 13 21:11:11.055550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055425    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.055810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 21:11:11.055971 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.055952    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 21:11:11.056097 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.056077    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.056350 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.056331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 21:11:11.056418 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.056378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.059370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.058953    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 21:11:11.059370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.059187    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 21:11:11.059370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.059233    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6fkct\""
Jul 13 21:11:11.059591 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.059457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:11.059591 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.059538    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:11.061388 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.061070    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.061388 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.061074    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:11:11.062418 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-cni-binary-copy\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062418 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-bin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-tmp\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.062571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-system-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-socket-dir-parent\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-conf-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062536    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kqs9\" (UniqueName: \"kubernetes.io/projected/49fe447d-9430-4e99-a587-f23eac1c8b03-kube-api-access-9kqs9\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6mxr\" (UniqueName: \"kubernetes.io/projected/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-kube-api-access-h6mxr\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062611    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-k8s-cni-cncf-io\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-multus\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-iptables-alerter-script\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bskkn\" (UniqueName: \"kubernetes.io/projected/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-kube-api-access-bskkn\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-kubelet-config\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.062851 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062813    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f544454-66d5-4dc8-834b-f818696cdbf4-konnectivity-ca\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-conf\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-tuned\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-host-slash\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.062999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysconfig\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-kubelet\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-hostroot\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063079    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-dbus\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063107    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f544454-66d5-4dc8-834b-f818696cdbf4-agent-certs\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-var-lib-kubelet\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-kubernetes\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-sys\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063288    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-lib-modules\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-host\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-netns\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-multus-certs\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-systemd\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5qvn\" (UniqueName: \"kubernetes.io/projected/aebe268f-9108-4eda-8f18-c7cdb841a81f-kube-api-access-g5qvn\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-os-release\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.063911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063805    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-daemon-config\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.065805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4m5k4\""
Jul 13 21:11:11.066264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063848    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.067616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.066676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.067616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.066774    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d8kvp\""
Jul 13 21:11:11.067616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.065951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 21:11:11.068677 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.063880    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.068677 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.067958    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.068677 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.068432    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.068954 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.068815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-etc-kubernetes\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.068954 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.068902    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-modprobe-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.068954 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.068935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-run\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.069115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.068966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-cnibin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.071401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.069252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 21:11:11.071401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.070166    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j498h\""
Jul 13 21:11:11.071401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.070187    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 21:11:11.071401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.070323    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 21:11:11.104346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.104318    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-94ckk"
Jul 13 21:11:11.119712 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.119684    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-94ckk"
Jul 13 21:11:11.151381 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.151289    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 21:11:11.169789 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169783    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0729b1c1-8389-4d40-ab63-87e52c39942e-tmp-dir\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqxw8\" (UniqueName: \"kubernetes.io/projected/23115f88-178a-4e5b-9c3f-34a160fb24e3-kube-api-access-vqxw8\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-netns\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169877    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-var-lib-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-ovn\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.169994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.169966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170003    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-kubernetes\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-lib-modules\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-multus-certs\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-os-release\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23115f88-178a-4e5b-9c3f-34a160fb24e3-serviceca\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-registration-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-modprobe-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-run\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-cnibin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170380    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-cni-binary-copy\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170406    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-bin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.170436 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170433    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23115f88-178a-4e5b-9c3f-34a160fb24e3-host\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170458    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-slash\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170513    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-tmp\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kqs9\" (UniqueName: \"kubernetes.io/projected/49fe447d-9430-4e99-a587-f23eac1c8b03-kube-api-access-9kqs9\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.170563    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-node-log\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-k8s-cni-cncf-io\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170623    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-lib-modules\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170627    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-multus\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.170658    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:11.670630295 +0000 UTC m=+3.087556643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-multus-certs\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170739    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-k8s-cni-cncf-io\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-cnibin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-kubernetes\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-modprobe-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170878    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-iptables-alerter-script\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.171267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170913    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bskkn\" (UniqueName: \"kubernetes.io/projected/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-kube-api-access-bskkn\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-multus\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-cni-bin\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170973    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-d\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.170991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-config\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-run\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f544454-66d5-4dc8-834b-f818696cdbf4-konnectivity-ca\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-system-cni-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171211    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nqdn\" (UniqueName: \"kubernetes.io/projected/f79973c8-85c1-4284-b467-3788edeb087a-kube-api-access-9nqdn\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-log-socket\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171408    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171414    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-socket-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171461    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-dbus\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-cni-binary-copy\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8lkq\" (UniqueName: \"kubernetes.io/projected/0729b1c1-8389-4d40-ab63-87e52c39942e-kube-api-access-p8lkq\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-kubelet\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92d6083b-543c-4039-8846-0c64da058fdc-ovn-node-metrics-cert\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.172166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-iptables-alerter-script\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-var-lib-kubelet\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-etc-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-dbus\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171629    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-env-overrides\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-var-lib-kubelet\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-sys-fs\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f724l\" (UniqueName: \"kubernetes.io/projected/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kube-api-access-f724l\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171768    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-binary-copy\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-sys\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171808    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6f544454-66d5-4dc8-834b-f818696cdbf4-konnectivity-ca\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-host\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171892    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-netns\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-sys\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-systemd\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171964    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5qvn\" (UniqueName: \"kubernetes.io/projected/aebe268f-9108-4eda-8f18-c7cdb841a81f-kube-api-access-g5qvn\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-run-netns\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.171988    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-os-release\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-daemon-config\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-host\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-systemd\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172055    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-etc-kubernetes\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-os-release\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172123    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-cnibin\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-device-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-etc-kubernetes\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-systemd-units\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172235    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-system-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-socket-dir-parent\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-conf-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-bin\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-system-cni-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.173880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172352    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-socket-dir-parent\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-script-lib\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcq2q\" (UniqueName: \"kubernetes.io/projected/92d6083b-543c-4039-8846-0c64da058fdc-kube-api-access-lcq2q\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-conf-dir\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6mxr\" (UniqueName: \"kubernetes.io/projected/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-kube-api-access-h6mxr\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-kubelet-config\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0729b1c1-8389-4d40-ab63-87e52c39942e-hosts-file\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-conf\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172584    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49fe447d-9430-4e99-a587-f23eac1c8b03-multus-daemon-config\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-tuned\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-host-slash\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.172685    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-host-slash\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7476560c-3a7d-42bc-88a6-7f2492d06a09-kubelet-config\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.172807    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:11.672785803 +0000 UTC m=+3.089712126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:11.174749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysctl-conf\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-systemd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysconfig\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-kubelet\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-hostroot\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.172988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.173011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-host-var-lib-kubelet\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.173020    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-netd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.173020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-sysconfig\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.173056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f544454-66d5-4dc8-834b-f818696cdbf4-agent-certs\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.173079    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49fe447d-9430-4e99-a587-f23eac1c8b03-hostroot\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.177271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.177142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6f544454-66d5-4dc8-834b-f818696cdbf4-agent-certs\") pod \"konnectivity-agent-26wqg\" (UID: \"6f544454-66d5-4dc8-834b-f818696cdbf4\") " pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.177858 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.177667    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-etc-tuned\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.180503 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.180481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aebe268f-9108-4eda-8f18-c7cdb841a81f-tmp\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.183818 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.183706    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal" event={"ID":"0ac4ccc85f99651906e4767386806ad9","Type":"ContainerStarted","Data":"42cccbc71d4325c22db31bc1403d974109f03ca5f7449fe9f0ca351a6a8b256a"}
Jul 13 21:11:11.184360 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.184199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5qvn\" (UniqueName: \"kubernetes.io/projected/aebe268f-9108-4eda-8f18-c7cdb841a81f-kube-api-access-g5qvn\") pod \"tuned-gs2rp\" (UID: \"aebe268f-9108-4eda-8f18-c7cdb841a81f\") " pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.185210 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.185165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kqs9\" (UniqueName: \"kubernetes.io/projected/49fe447d-9430-4e99-a587-f23eac1c8b03-kube-api-access-9kqs9\") pod \"multus-7zgs6\" (UID: \"49fe447d-9430-4e99-a587-f23eac1c8b03\") " pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.185717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.185536    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal" event={"ID":"329fa49187de2fbb90a0ad7755dac966","Type":"ContainerStarted","Data":"47daf8ec3c75fc6ee0efcca8da0d51d98b58147a91869de74d66fbcc63622883"}
Jul 13 21:11:11.185717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.185685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bskkn\" (UniqueName: \"kubernetes.io/projected/ff5d6303-d56a-4e49-8ffa-b4ea4306411f-kube-api-access-bskkn\") pod \"iptables-alerter-p5t77\" (UID: \"ff5d6303-d56a-4e49-8ffa-b4ea4306411f\") " pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.187420 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.187310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6mxr\" (UniqueName: \"kubernetes.io/projected/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-kube-api-access-h6mxr\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273517    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0729b1c1-8389-4d40-ab63-87e52c39942e-hosts-file\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-systemd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-netd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0729b1c1-8389-4d40-ab63-87e52c39942e-tmp-dir\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqxw8\" (UniqueName: \"kubernetes.io/projected/23115f88-178a-4e5b-9c3f-34a160fb24e3-kube-api-access-vqxw8\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-netns\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-var-lib-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-ovn\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273858    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273888    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-os-release\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23115f88-178a-4e5b-9c3f-34a160fb24e3-serviceca\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273918    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.275182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-registration-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.273991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-systemd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23115f88-178a-4e5b-9c3f-34a160fb24e3-host\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-slash\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274122    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-node-log\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0729b1c1-8389-4d40-ab63-87e52c39942e-hosts-file\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-os-release\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-netns\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-netd\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-run-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274152    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-config\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/23115f88-178a-4e5b-9c3f-34a160fb24e3-host\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-system-cni-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-node-log\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.276113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274490    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-slash\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nqdn\" (UniqueName: \"kubernetes.io/projected/f79973c8-85c1-4284-b467-3788edeb087a-kube-api-access-9nqdn\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-system-cni-dir\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274534    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-log-socket\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-socket-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8lkq\" (UniqueName: \"kubernetes.io/projected/0729b1c1-8389-4d40-ab63-87e52c39942e-kube-api-access-p8lkq\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-kubelet\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-etc-selinux\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92d6083b-543c-4039-8846-0c64da058fdc-ovn-node-metrics-cert\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-etc-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-env-overrides\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-sys-fs\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274703    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-registration-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f724l\" (UniqueName: \"kubernetes.io/projected/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kube-api-access-f724l\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-var-lib-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/23115f88-178a-4e5b-9c3f-34a160fb24e3-serviceca\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.277014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274450    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-binary-copy\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-config\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274781    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-cnibin\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-device-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-systemd-units\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-bin\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-socket-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274914    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274950    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0729b1c1-8389-4d40-ab63-87e52c39942e-tmp-dir\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274782    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-kubelet\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-run-ovn\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275110    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275187    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-device-dir\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275192    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-etc-openvswitch\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.277780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-script-lib\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-systemd-units\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275243    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcq2q\" (UniqueName: \"kubernetes.io/projected/92d6083b-543c-4039-8846-0c64da058fdc-kube-api-access-lcq2q\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-host-cni-bin\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.274996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92d6083b-543c-4039-8846-0c64da058fdc-log-socket\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-sys-fs\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-env-overrides\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92d6083b-543c-4039-8846-0c64da058fdc-ovnkube-script-lib\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f79973c8-85c1-4284-b467-3788edeb087a-cnibin\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.278686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.275672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f79973c8-85c1-4284-b467-3788edeb087a-cni-binary-copy\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.279554 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.279528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92d6083b-543c-4039-8846-0c64da058fdc-ovn-node-metrics-cert\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.288906 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.288876    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:11.288906 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.288908    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:11.289122 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.288921    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:11.289122 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.288989    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:11.788969615 +0000 UTC m=+3.205895962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:11.292873 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.292843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8lkq\" (UniqueName: \"kubernetes.io/projected/0729b1c1-8389-4d40-ab63-87e52c39942e-kube-api-access-p8lkq\") pod \"node-resolver-zngbn\" (UID: \"0729b1c1-8389-4d40-ab63-87e52c39942e\") " pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.293015 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.292894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f724l\" (UniqueName: \"kubernetes.io/projected/d371c4d4-c1d7-4b3d-9a0f-12f087f39eca-kube-api-access-f724l\") pod \"aws-ebs-csi-driver-node-rcrwv\" (UID: \"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.293289 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.293263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcq2q\" (UniqueName: \"kubernetes.io/projected/92d6083b-543c-4039-8846-0c64da058fdc-kube-api-access-lcq2q\") pod \"ovnkube-node-v6skj\" (UID: \"92d6083b-543c-4039-8846-0c64da058fdc\") " pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.293763 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.293733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqxw8\" (UniqueName: \"kubernetes.io/projected/23115f88-178a-4e5b-9c3f-34a160fb24e3-kube-api-access-vqxw8\") pod \"node-ca-98x85\" (UID: \"23115f88-178a-4e5b-9c3f-34a160fb24e3\") " pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.294342 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.294307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nqdn\" (UniqueName: \"kubernetes.io/projected/f79973c8-85c1-4284-b467-3788edeb087a-kube-api-access-9nqdn\") pod \"multus-additional-cni-plugins-9msh7\" (UID: \"f79973c8-85c1-4284-b467-3788edeb087a\") " pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.359626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.359574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7zgs6"
Jul 13 21:11:11.371399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.371368    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p5t77"
Jul 13 21:11:11.372262 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.372229    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49fe447d_9430_4e99_a587_f23eac1c8b03.slice/crio-5404edff284b4df4a6ede0c52fca01a41bc565fd034208937157ec52de4adcaa WatchSource:0}: Error finding container 5404edff284b4df4a6ede0c52fca01a41bc565fd034208937157ec52de4adcaa: Status 404 returned error can't find the container with id 5404edff284b4df4a6ede0c52fca01a41bc565fd034208937157ec52de4adcaa
Jul 13 21:11:11.382819 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.382783    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5d6303_d56a_4e49_8ffa_b4ea4306411f.slice/crio-5184c526bc8628d97bda16cc1e7d598f83b1e9937240ce94c747009bab49d399 WatchSource:0}: Error finding container 5184c526bc8628d97bda16cc1e7d598f83b1e9937240ce94c747009bab49d399: Status 404 returned error can't find the container with id 5184c526bc8628d97bda16cc1e7d598f83b1e9937240ce94c747009bab49d399
Jul 13 21:11:11.384338 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.384307    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:11.390569 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.390545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gs2rp"
Jul 13 21:11:11.394303 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.394270    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f544454_66d5_4dc8_834b_f818696cdbf4.slice/crio-f1b97d5f9a9fc8c217a486c443abc336cb1b469ff5a74bbd406a0ade06e30329 WatchSource:0}: Error finding container f1b97d5f9a9fc8c217a486c443abc336cb1b469ff5a74bbd406a0ade06e30329: Status 404 returned error can't find the container with id f1b97d5f9a9fc8c217a486c443abc336cb1b469ff5a74bbd406a0ade06e30329
Jul 13 21:11:11.399364 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.399339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:11.400439 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.400383    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaebe268f_9108_4eda_8f18_c7cdb841a81f.slice/crio-a57ccdd937c3eec37573304b65e3a4b4ecdb1e466e82b6a8d93e6b493a22a5c6 WatchSource:0}: Error finding container a57ccdd937c3eec37573304b65e3a4b4ecdb1e466e82b6a8d93e6b493a22a5c6: Status 404 returned error can't find the container with id a57ccdd937c3eec37573304b65e3a4b4ecdb1e466e82b6a8d93e6b493a22a5c6
Jul 13 21:11:11.408879 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.408803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9msh7"
Jul 13 21:11:11.410893 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.410859    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d6083b_543c_4039_8846_0c64da058fdc.slice/crio-0995542967c2d78090673e73d5c18179ec7b787ecef25f891a055dd01e2395be WatchSource:0}: Error finding container 0995542967c2d78090673e73d5c18179ec7b787ecef25f891a055dd01e2395be: Status 404 returned error can't find the container with id 0995542967c2d78090673e73d5c18179ec7b787ecef25f891a055dd01e2395be
Jul 13 21:11:11.418890 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.418863    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv"
Jul 13 21:11:11.419286 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.419258    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79973c8_85c1_4284_b467_3788edeb087a.slice/crio-9d4629e993cb1edd7cafff2b49c634ec74c4b4ee7edf5ca1e3a3139846ebde2e WatchSource:0}: Error finding container 9d4629e993cb1edd7cafff2b49c634ec74c4b4ee7edf5ca1e3a3139846ebde2e: Status 404 returned error can't find the container with id 9d4629e993cb1edd7cafff2b49c634ec74c4b4ee7edf5ca1e3a3139846ebde2e
Jul 13 21:11:11.425522 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.425430    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zngbn"
Jul 13 21:11:11.429490 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:11.429457    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd371c4d4_c1d7_4b3d_9a0f_12f087f39eca.slice/crio-a4e0e0eb379ec7fa26fdcaf93acc6c90086c22257fdf8929ac3d2881d40beeff WatchSource:0}: Error finding container a4e0e0eb379ec7fa26fdcaf93acc6c90086c22257fdf8929ac3d2881d40beeff: Status 404 returned error can't find the container with id a4e0e0eb379ec7fa26fdcaf93acc6c90086c22257fdf8929ac3d2881d40beeff
Jul 13 21:11:11.433248 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.433220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-98x85"
Jul 13 21:11:11.678673 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.678569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:11.678866 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.678684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:11.678866 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.678736    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:11.678866 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.678824    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:11.678866 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.678832    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:12.678815593 +0000 UTC m=+4.095741917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:11.679086 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.678884    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:12.678866711 +0000 UTC m=+4.095793042 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:11.879707 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:11.879665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:11.879914 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.879854    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:11.879914 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.879875    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:11.879914 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.879886    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:11.880035 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:11.879945    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:12.879929841 +0000 UTC m=+4.296856186 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:12.123201 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.123090    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:06:11 +0000 UTC" deadline="2028-04-10 16:40:51.49062715 +0000 UTC"
Jul 13 21:11:12.123201 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.123138    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15283h29m39.36749395s"
Jul 13 21:11:12.188721 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.188682    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerStarted","Data":"9d4629e993cb1edd7cafff2b49c634ec74c4b4ee7edf5ca1e3a3139846ebde2e"}
Jul 13 21:11:12.189935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.189906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gs2rp" event={"ID":"aebe268f-9108-4eda-8f18-c7cdb841a81f","Type":"ContainerStarted","Data":"a57ccdd937c3eec37573304b65e3a4b4ecdb1e466e82b6a8d93e6b493a22a5c6"}
Jul 13 21:11:12.191147 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.191121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-26wqg" event={"ID":"6f544454-66d5-4dc8-834b-f818696cdbf4","Type":"ContainerStarted","Data":"f1b97d5f9a9fc8c217a486c443abc336cb1b469ff5a74bbd406a0ade06e30329"}
Jul 13 21:11:12.192344 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.192319    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p5t77" event={"ID":"ff5d6303-d56a-4e49-8ffa-b4ea4306411f","Type":"ContainerStarted","Data":"5184c526bc8628d97bda16cc1e7d598f83b1e9937240ce94c747009bab49d399"}
Jul 13 21:11:12.193509 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.193487    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zgs6" event={"ID":"49fe447d-9430-4e99-a587-f23eac1c8b03","Type":"ContainerStarted","Data":"5404edff284b4df4a6ede0c52fca01a41bc565fd034208937157ec52de4adcaa"}
Jul 13 21:11:12.194687 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.194665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv" event={"ID":"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca","Type":"ContainerStarted","Data":"a4e0e0eb379ec7fa26fdcaf93acc6c90086c22257fdf8929ac3d2881d40beeff"}
Jul 13 21:11:12.195989 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.195950    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"0995542967c2d78090673e73d5c18179ec7b787ecef25f891a055dd01e2395be"}
Jul 13 21:11:12.687005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.686967    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:12.687220 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.687049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:12.687288 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.687207    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:12.687288 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.687263    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:12.687401 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.687300    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:14.687273645 +0000 UTC m=+6.104199981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:12.687401 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.687325    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:14.687314401 +0000 UTC m=+6.104240725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:12.694771 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:12.694722    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0729b1c1_8389_4d40_ab63_87e52c39942e.slice/crio-58d40265bce1c82b24fbda7d33e391f3e793d163c198f0675a5b05e90f27c472 WatchSource:0}: Error finding container 58d40265bce1c82b24fbda7d33e391f3e793d163c198f0675a5b05e90f27c472: Status 404 returned error can't find the container with id 58d40265bce1c82b24fbda7d33e391f3e793d163c198f0675a5b05e90f27c472
Jul 13 21:11:12.718330 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:12.718299    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23115f88_178a_4e5b_9c3f_34a160fb24e3.slice/crio-86afa916f03ce4e5e367508a4da5c9d5c331cd691ccb13388d02cc1e675b90ab WatchSource:0}: Error finding container 86afa916f03ce4e5e367508a4da5c9d5c331cd691ccb13388d02cc1e675b90ab: Status 404 returned error can't find the container with id 86afa916f03ce4e5e367508a4da5c9d5c331cd691ccb13388d02cc1e675b90ab
Jul 13 21:11:12.889286 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:12.889011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:12.889492 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.889328    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:12.889492 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.889363    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:12.889492 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.889380    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:12.889492 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:12.889469    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:14.889446948 +0000 UTC m=+6.306373287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:13.123489 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.123338    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:06:11 +0000 UTC" deadline="2028-04-04 12:40:26.603043692 +0000 UTC"
Jul 13 21:11:13.123489 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.123406    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15135h29m13.47964307s"
Jul 13 21:11:13.180187 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.179528    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:13.180187 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:13.179685    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:13.180187 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.180153    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:13.180477 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:13.180249    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:13.180477 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.180281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:13.180477 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:13.180353    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:13.207044 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.206996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zngbn" event={"ID":"0729b1c1-8389-4d40-ab63-87e52c39942e","Type":"ContainerStarted","Data":"58d40265bce1c82b24fbda7d33e391f3e793d163c198f0675a5b05e90f27c472"}
Jul 13 21:11:13.215666 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.215356    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal" event={"ID":"0ac4ccc85f99651906e4767386806ad9","Type":"ContainerStarted","Data":"e449dee7e3f09b147c6e666fdb28639c8a2950b1fcfcb686295ef4e190bf3a1f"}
Jul 13 21:11:13.220169 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:13.219907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98x85" event={"ID":"23115f88-178a-4e5b-9c3f-34a160fb24e3","Type":"ContainerStarted","Data":"86afa916f03ce4e5e367508a4da5c9d5c331cd691ccb13388d02cc1e675b90ab"}
Jul 13 21:11:14.239512 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.239469    2577 generic.go:356] "Generic (PLEG): container finished" podID="329fa49187de2fbb90a0ad7755dac966" containerID="52cd4f89da0a923b66b9d6b2c9392f909a84bd9a0725789061aea9294325fea2" exitCode=0
Jul 13 21:11:14.240347 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.240098    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal" event={"ID":"329fa49187de2fbb90a0ad7755dac966","Type":"ContainerDied","Data":"52cd4f89da0a923b66b9d6b2c9392f909a84bd9a0725789061aea9294325fea2"}
Jul 13 21:11:14.257640 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.256656    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-242.ec2.internal" podStartSLOduration=4.256633475 podStartE2EDuration="4.256633475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:13.232499604 +0000 UTC m=+4.649425951" watchObservedRunningTime="2026-07-13 21:11:14.256633475 +0000 UTC m=+5.673559826"
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.703045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.703118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.703258    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.703328    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:18.703307203 +0000 UTC m=+10.120233530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.703784    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:14.703976 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.703859    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:18.703826906 +0000 UTC m=+10.120753244 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:14.904418 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:14.904197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:14.904623 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.904428    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:14.904623 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.904449    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:14.904623 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.904464    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:14.904623 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:14.904533    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:18.90451165 +0000 UTC m=+10.321437976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:15.177979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:15.177872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:15.178137 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:15.177881    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:15.178137 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:15.178015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:15.178137 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:15.178008    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:15.178137 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:15.178090    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:15.178348 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:15.178145    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:15.258605 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:15.258560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal" event={"ID":"329fa49187de2fbb90a0ad7755dac966","Type":"ContainerStarted","Data":"e9104d2b1ad34c961b5fda25484bfa30ff2748f046370ceac1b793da0a6cef76"}
Jul 13 21:11:15.273312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:15.273254    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-242.ec2.internal" podStartSLOduration=5.2732314240000004 podStartE2EDuration="5.273231424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:15.272511128 +0000 UTC m=+6.689437479" watchObservedRunningTime="2026-07-13 21:11:15.273231424 +0000 UTC m=+6.690157770"
Jul 13 21:11:17.177967 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:17.177935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:17.178437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:17.178007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:17.178437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:17.178012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:17.178437 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:17.178105    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:17.178437 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:17.178196    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:17.178437 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:17.178309    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:18.740122 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:18.740000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:18.740122 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:18.740063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:18.740658 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.740166    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:18.740658 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.740237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:26.7402169 +0000 UTC m=+18.157143225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:18.740658 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.740176    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:18.740658 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.740319    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:26.74030112 +0000 UTC m=+18.157227444 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:18.941463 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:18.941426    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:18.941652 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.941610    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:18.941652 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.941635    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:18.941652 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.941648    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:18.941811 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:18.941710    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:26.941690399 +0000 UTC m=+18.358616736 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:19.178206 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:19.178123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:19.178370 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:19.178237    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:19.178370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:19.178289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:19.178370 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:19.178352    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:19.178526 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:19.178472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:19.178619 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:19.178596    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:21.177145 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:21.177113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:21.177608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:21.177113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:21.177608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:21.177251    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:21.177608 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:21.177244    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:21.177608 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:21.177334    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:21.177608 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:21.177443    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:23.177054 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:23.177014    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:23.177507 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:23.177019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:23.177507 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:23.177151    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:23.177507 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:23.177019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:23.177507 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:23.177237    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:23.177507 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:23.177377    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:25.177320 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:25.177273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:25.177753 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:25.177280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:25.177753 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:25.177402    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:25.177753 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:25.177280    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:25.177753 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:25.177494    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:25.177753 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:25.177591    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:26.806409 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:26.806327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:26.806409 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:26.806402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:26.806912 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:26.806503    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:26.806912 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:26.806561    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:42.806548598 +0000 UTC m=+34.223474924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:26.806912 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:26.806503    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:26.806912 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:26.806631    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:11:42.806618196 +0000 UTC m=+34.223544534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:27.007457 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:27.007410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:27.007638 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.007580    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:27.007638 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.007599    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:27.007638 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.007609    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:27.007845 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.007672    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:43.007646695 +0000 UTC m=+34.424573038 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:27.177400 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:27.177318    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:27.177577 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:27.177318    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:27.177577 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.177451    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:27.177577 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.177548    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:27.177577 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:27.177327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:27.177806 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:27.177664    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:29.177935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:29.177893    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:29.178399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:29.177999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:29.178399 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:29.178005    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:29.178399 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:29.178125    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:29.178399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:29.178140    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:29.178399 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:29.178229    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:31.177123 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.177031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:31.177490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.177038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:31.177490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.177145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:31.177490 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:31.177168    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:31.177490 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:31.177223    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:31.177490 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:31.177309    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:31.290499 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.290423    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-98x85" event={"ID":"23115f88-178a-4e5b-9c3f-34a160fb24e3","Type":"ContainerStarted","Data":"af37f5bc0ba629810ba368beb694ccdc3da01ad9a9cba102c2f97987b78c74a5"}
Jul 13 21:11:31.293424 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.292988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zngbn" event={"ID":"0729b1c1-8389-4d40-ab63-87e52c39942e","Type":"ContainerStarted","Data":"670a4265fb51ce570a12bef277d899f6c5f4bad485b8852664b40d58c0f30e3c"}
Jul 13 21:11:31.296701 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.296637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gs2rp" event={"ID":"aebe268f-9108-4eda-8f18-c7cdb841a81f","Type":"ContainerStarted","Data":"eede22d4922e83adb97d87b9c2e992ebe8d33cbe2de6e7d643e7b4157d79a20c"}
Jul 13 21:11:31.298782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.298731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-26wqg" event={"ID":"6f544454-66d5-4dc8-834b-f818696cdbf4","Type":"ContainerStarted","Data":"84f2546d05559dd05efb16ef8f72fa209c0e2b28eee7380512e51803cd3f886d"}
Jul 13 21:11:31.306443 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.305678    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-98x85" podStartSLOduration=4.083237574 podStartE2EDuration="22.305631338s" totalImagesPullingTime="18.222393764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:31.304932725 +0000 UTC m=+22.721859066" watchObservedRunningTime="2026-07-13 21:11:31.305631338 +0000 UTC m=+22.722557685"
Jul 13 21:11:31.319217 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.319093    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-26wqg" podStartSLOduration=2.772738953 podStartE2EDuration="22.319078281s" totalImagesPullingTime="19.546339328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:31.318357948 +0000 UTC m=+22.735284291" watchObservedRunningTime="2026-07-13 21:11:31.319078281 +0000 UTC m=+22.736004626"
Jul 13 21:11:31.351815 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.351772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gs2rp" podStartSLOduration=2.8094820069999997 podStartE2EDuration="22.351742502s" totalImagesPullingTime="19.542260495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:31.337786771 +0000 UTC m=+22.754713115" watchObservedRunningTime="2026-07-13 21:11:31.351742502 +0000 UTC m=+22.768668847"
Jul 13 21:11:31.352166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:31.352139    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zngbn" podStartSLOduration=4.107893048 podStartE2EDuration="22.352130096s" totalImagesPullingTime="18.244237048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:31.351437144 +0000 UTC m=+22.768363489" watchObservedRunningTime="2026-07-13 21:11:31.352130096 +0000 UTC m=+22.769056440"
Jul 13 21:11:32.302479 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.302196    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="cfac380a928b32f767ce1368b8af8195f46cec3041044596c72e49b7c08a5184" exitCode=0
Jul 13 21:11:32.303308 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.302285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"cfac380a928b32f767ce1368b8af8195f46cec3041044596c72e49b7c08a5184"}
Jul 13 21:11:32.304038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.304014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7zgs6" event={"ID":"49fe447d-9430-4e99-a587-f23eac1c8b03","Type":"ContainerStarted","Data":"ca8ba1de51809473889d65e611505086a43d1397e9a0589ba9807c294e75da8d"}
Jul 13 21:11:32.305292 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.305270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv" event={"ID":"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca","Type":"ContainerStarted","Data":"59f13f204be6ba7ef229190c805150c4369020063158f3e40169332562102cbc"}
Jul 13 21:11:32.307977 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.307958    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:11:32.308305 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308287    2577 generic.go:356] "Generic (PLEG): container finished" podID="92d6083b-543c-4039-8846-0c64da058fdc" containerID="96484dc76adccfbed5aca8e2505e6c745e2e4e4ad359aec497ed98738339b51d" exitCode=1
Jul 13 21:11:32.308401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"47f2009ce08165999e29fff03c20c89b6fe78c42e19aa896e2c76b819c94c0f1"}
Jul 13 21:11:32.308458 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"f847d428f47db81f62a75309d28c603acb9ab94d307b1f3e36cea8d401623282"}
Jul 13 21:11:32.308493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"8cfee7fd3ac09364c4db13ce97cc1b3ace8728c54c51ad79bb214be3dc8194d2"}
Jul 13 21:11:32.308524 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308512    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"469e9d7de9a2f19aedb80c775811371ecddc425c9545b067418baa2108f310fe"}
Jul 13 21:11:32.308555 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerDied","Data":"96484dc76adccfbed5aca8e2505e6c745e2e4e4ad359aec497ed98738339b51d"}
Jul 13 21:11:32.308587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.308572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"5a127506d03409e5da2ef0fb47823a0377d54dacf020e5c763b01a5b9b2beaba"}
Jul 13 21:11:32.340702 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.340644    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7zgs6" podStartSLOduration=3.294823394 podStartE2EDuration="23.340627956s" totalImagesPullingTime="20.045804562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:32.340111665 +0000 UTC m=+23.757038008" watchObservedRunningTime="2026-07-13 21:11:32.340627956 +0000 UTC m=+23.757554299"
Jul 13 21:11:32.571746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:32.571719    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 21:11:33.129024 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.128915    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:11:32.571738424Z","UUID":"bd2a0926-54ec-4ff2-8b2b-998ae9b291bf","Handler":null,"Name":"","Endpoint":""}
Jul 13 21:11:33.130999 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.130897    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 21:11:33.130999 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.130935    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 21:11:33.177115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.177072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:33.177115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.177096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:33.177362 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:33.177224    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:33.177362 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.177259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:33.177362 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:33.177350    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:33.177537 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:33.177431    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:33.312777 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.312652    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p5t77" event={"ID":"ff5d6303-d56a-4e49-8ffa-b4ea4306411f","Type":"ContainerStarted","Data":"b8f725d2566be0e40f91c7328c48449ed7f602c6617b015857626f459ad41d43"}
Jul 13 21:11:33.314943 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.314917    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv" event={"ID":"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca","Type":"ContainerStarted","Data":"a244c7d9c3330d168222d0c9e178869fa978f0c5d24b47eaf622995f6319f562"}
Jul 13 21:11:33.350970 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:33.350906    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p5t77" podStartSLOduration=4.791140849 podStartE2EDuration="24.350883571s" totalImagesPullingTime="19.559742722s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:33.350441559 +0000 UTC m=+24.767367927" watchObservedRunningTime="2026-07-13 21:11:33.350883571 +0000 UTC m=+24.767809916"
Jul 13 21:11:34.321038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:34.320132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv" event={"ID":"d371c4d4-c1d7-4b3d-9a0f-12f087f39eca","Type":"ContainerStarted","Data":"9e264b52647520330d020b55147f2c2271fb6d0b101994dc8940fa1c46030734"}
Jul 13 21:11:34.325691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:34.325664    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:11:34.326185 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:34.326159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"eb952901837f5030606059e1937448e74489a11bdd37cb169476e96f33eda603"}
Jul 13 21:11:34.339393 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:34.339345    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rcrwv" podStartSLOduration=3.647385032 podStartE2EDuration="25.339329988s" totalImagesPullingTime="21.691944956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:34.338724469 +0000 UTC m=+25.755650814" watchObservedRunningTime="2026-07-13 21:11:34.339329988 +0000 UTC m=+25.756256331"
Jul 13 21:11:35.177679 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:35.177647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:35.177896 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:35.177647    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:35.177896 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:35.177805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:35.177896 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:35.177798    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:35.178057 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:35.177903    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:35.178057 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:35.178000    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:35.691579 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:35.691545    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:35.692333 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:35.692310    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:36.329682 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:36.329656    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:36.330449 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:36.330430    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-26wqg"
Jul 13 21:11:37.177796 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.177600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:37.178330 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.177606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:37.178330 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:37.177878    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:37.178330 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:37.177990    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:37.178330 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.177622    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:37.178330 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:37.178093    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:37.335091 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.335069    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:11:37.335461 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.335432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"310a420b328b8a94c3f8884b9d0e4d6e779c58b34d3a3f643cbc0a28c115c97b"}
Jul 13 21:11:37.335718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.335693    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:37.335916 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.335889    2577 scope.go:117] "RemoveContainer" containerID="96484dc76adccfbed5aca8e2505e6c745e2e4e4ad359aec497ed98738339b51d"
Jul 13 21:11:37.337948 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.337924    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="7483d26133cfa7b25e196ed59053d8286f74a719c54176f04138890f6f83f1a6" exitCode=0
Jul 13 21:11:37.338043 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.338004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"7483d26133cfa7b25e196ed59053d8286f74a719c54176f04138890f6f83f1a6"}
Jul 13 21:11:37.352745 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:37.352719    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:38.344671 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.344510    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:11:38.345136 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.345049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" event={"ID":"92d6083b-543c-4039-8846-0c64da058fdc","Type":"ContainerStarted","Data":"d8006b74ddce69758b7567f444215bc84b0ef27b1742b2d6fd4384b77fa9c311"}
Jul 13 21:11:38.345243 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.345230    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:38.345288 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.345281    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:38.362664 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.362637    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:11:38.375856 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.375748    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjjbg"]
Jul 13 21:11:38.376029 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.375960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:38.376125 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:38.376098    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:38.376465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.376437    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27v2r"]
Jul 13 21:11:38.376561 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.376533    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:38.376645 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:38.376624    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:38.377142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.377103    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj" podStartSLOduration=9.544420006 podStartE2EDuration="29.377086773s" totalImagesPullingTime="19.832666767s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:38.375377605 +0000 UTC m=+29.792303989" watchObservedRunningTime="2026-07-13 21:11:38.377086773 +0000 UTC m=+29.794013118"
Jul 13 21:11:38.381264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.379026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzds7"]
Jul 13 21:11:38.381264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:38.379181    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:38.381264 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:38.379325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:39.349452 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:39.349418    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="ea14738f66a62756265e89783086737474c25d94d1fcc880c98368a275318c25" exitCode=0
Jul 13 21:11:39.349990 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:39.349498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"ea14738f66a62756265e89783086737474c25d94d1fcc880c98368a275318c25"}
Jul 13 21:11:40.176833 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:40.176792    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:40.176833 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:40.176834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:40.177061 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:40.176834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:40.177061 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:40.176918    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:40.177061 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:40.177025    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:40.177184 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:40.177132    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:41.356415 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:41.356378    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="dcc7631904673b8c0718386c36152e537c9bb800f1b0c627aa3e3a45ab00cbcd" exitCode=0
Jul 13 21:11:41.357124 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:41.356449    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"dcc7631904673b8c0718386c36152e537c9bb800f1b0c627aa3e3a45ab00cbcd"}
Jul 13 21:11:42.177208 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:42.177135    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:42.177208 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:42.177172    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:42.177448 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:42.177260    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:42.177448 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.177269    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27v2r" podUID="54094210-e8fb-4c14-95da-7ab6dd6d35a5"
Jul 13 21:11:42.177448 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.177422    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bjjbg" podUID="7476560c-3a7d-42bc-88a6-7f2492d06a09"
Jul 13 21:11:42.177542 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.177513    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rzds7" podUID="fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab"
Jul 13 21:11:42.824992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:42.824685    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:42.824992 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.824861    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:42.825464 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.825021    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs podName:fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab nodeName:}" failed. No retries permitted until 2026-07-13 21:12:14.824996052 +0000 UTC m=+66.241922376 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs") pod "network-metrics-daemon-rzds7" (UID: "fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:42.825464 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:42.825121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:42.825464 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.825233    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:42.825464 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:42.825272    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret podName:7476560c-3a7d-42bc-88a6-7f2492d06a09 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:14.82526217 +0000 UTC m=+66.242188492 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret") pod "global-pull-secret-syncer-bjjbg" (UID: "7476560c-3a7d-42bc-88a6-7f2492d06a09") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:43.026379 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:43.026342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:43.026569 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:43.026529    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:43.026569 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:43.026553    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:43.026569 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:43.026566    2577 projected.go:196] Error preparing data for projected volume kube-api-access-8gxvp for pod openshift-network-diagnostics/network-check-target-27v2r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:43.026723 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:43.026631    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp podName:54094210-e8fb-4c14-95da-7ab6dd6d35a5 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:15.026611242 +0000 UTC m=+66.443537578 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8gxvp" (UniqueName: "kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp") pod "network-check-target-27v2r" (UID: "54094210-e8fb-4c14-95da-7ab6dd6d35a5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:44.152230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.152202    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-242.ec2.internal" event="NodeReady"
Jul 13 21:11:44.152703 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.152371    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 21:11:44.177606 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.177574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:11:44.177803 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.177574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:11:44.178022 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.177584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:11:44.179972 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.179943    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t2zb8\""
Jul 13 21:11:44.180397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.180269    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:11:44.180397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.180285    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:11:44.180397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.180272    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:11:44.180397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.180298    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vxg2k\""
Jul 13 21:11:44.180660 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.180508    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:11:44.189418 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.189395    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69bffb78b7-r2865"]
Jul 13 21:11:44.191823 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.191803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.194370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.194327    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 13 21:11:44.194370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.194343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 13 21:11:44.194370 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.194332    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 13 21:11:44.194652 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.194635    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2ksct\""
Jul 13 21:11:44.198234 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.198208    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kqxx5"]
Jul 13 21:11:44.202608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.200696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.202608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.201856    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8w765"]
Jul 13 21:11:44.203271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.203253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 13 21:11:44.203429 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.203366    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 21:11:44.203429 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.203384    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-btz4g\""
Jul 13 21:11:44.203535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.203505    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 21:11:44.204606 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.204164    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.206549 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.206407    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 21:11:44.206549 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.206463    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gcwv8\""
Jul 13 21:11:44.206706 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.206619    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 21:11:44.206706 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.206660    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 21:11:44.206812 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.206729    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69bffb78b7-r2865"]
Jul 13 21:11:44.217688 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.217627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kqxx5"]
Jul 13 21:11:44.218606 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.218584    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8w765"]
Jul 13 21:11:44.334870 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.334830    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.334870 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.334866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/edb8e8a1-8d39-4f54-9858-158909fbd6fb-cert\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.335106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.334887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp86p\" (UniqueName: \"kubernetes.io/projected/edb8e8a1-8d39-4f54-9858-158909fbd6fb-kube-api-access-jp86p\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.335106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.334911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.334965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335051    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7c927257-e779-4148-a577-c9bec9ad63f2-config-volume\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.335312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vts2\" (UniqueName: \"kubernetes.io/projected/7c927257-e779-4148-a577-c9bec9ad63f2-kube-api-access-9vts2\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.335312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c927257-e779-4148-a577-c9bec9ad63f2-tmp-dir\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.335312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335543 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2vlr\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335543 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.335543 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.335452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.436901 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.436901 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/edb8e8a1-8d39-4f54-9858-158909fbd6fb-cert\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.436901 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp86p\" (UniqueName: \"kubernetes.io/projected/edb8e8a1-8d39-4f54-9858-158909fbd6fb-kube-api-access-jp86p\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.436970    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7c927257-e779-4148-a577-c9bec9ad63f2-config-volume\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.436983    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vts2\" (UniqueName: \"kubernetes.io/projected/7c927257-e779-4148-a577-c9bec9ad63f2-kube-api-access-9vts2\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.437069    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:44.937047713 +0000 UTC m=+36.353974038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c927257-e779-4148-a577-c9bec9ad63f2-tmp-dir\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.437176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2vlr\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437302    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437939 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437879    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c927257-e779-4148-a577-c9bec9ad63f2-tmp-dir\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.437939 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.437939 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.437917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7c927257-e779-4148-a577-c9bec9ad63f2-config-volume\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.438169 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.438150    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:11:44.438229 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.438170    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:11:44.438284 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.438235    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:11:44.938216054 +0000 UTC m=+36.355142378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:11:44.438342 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.438295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.442384 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.442344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.442384 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.442356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.442550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.442436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/edb8e8a1-8d39-4f54-9858-158909fbd6fb-cert\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.445402 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.445368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.445502 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.445429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp86p\" (UniqueName: \"kubernetes.io/projected/edb8e8a1-8d39-4f54-9858-158909fbd6fb-kube-api-access-jp86p\") pod \"ingress-canary-8w765\" (UID: \"edb8e8a1-8d39-4f54-9858-158909fbd6fb\") " pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.445833 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.445815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vts2\" (UniqueName: \"kubernetes.io/projected/7c927257-e779-4148-a577-c9bec9ad63f2-kube-api-access-9vts2\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.446290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.446270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2vlr\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.530302 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.530265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8w765"
Jul 13 21:11:44.941204 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.941165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:44.941406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:44.941244    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:44.941406 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.941307    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:11:44.941406 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.941331    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:11:44.941406 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.941388    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:11:44.941558 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.941396    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:11:45.941377392 +0000 UTC m=+37.358303715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:11:44.941558 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:44.941462    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:45.94144614 +0000 UTC m=+37.358372462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:11:45.035265 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.035212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:45.170439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.170402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:45.259662 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.259569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:45.360323 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.360290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:45.708981 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.708910    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8w765"]
Jul 13 21:11:45.714975 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:45.714947    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb8e8a1_8d39_4f54_9858_158909fbd6fb.slice/crio-cdebad9feb1630a25f4f41a711369a91d826dbd02745cae7b79e49b7840afee5 WatchSource:0}: Error finding container cdebad9feb1630a25f4f41a711369a91d826dbd02745cae7b79e49b7840afee5: Status 404 returned error can't find the container with id cdebad9feb1630a25f4f41a711369a91d826dbd02745cae7b79e49b7840afee5
Jul 13 21:11:45.950005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.949973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:45.950209 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:45.950043    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:45.950209 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:45.950138    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:11:45.950209 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:45.950156    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:11:45.950209 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:45.950160    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:11:45.950359 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:45.950220    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:11:47.950202649 +0000 UTC m=+39.367128971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:11:45.950359 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:45.950235    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:47.950229275 +0000 UTC m=+39.367155596 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:11:46.370551 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:46.370458    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8w765" event={"ID":"edb8e8a1-8d39-4f54-9858-158909fbd6fb","Type":"ContainerStarted","Data":"cdebad9feb1630a25f4f41a711369a91d826dbd02745cae7b79e49b7840afee5"}
Jul 13 21:11:46.376774 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:46.376732    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="a3e1e2a7021a1eb62d681bda195b180ca775e63c5fc6cb378cbbedd83c28b7b0" exitCode=0
Jul 13 21:11:46.376918 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:46.376793    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"a3e1e2a7021a1eb62d681bda195b180ca775e63c5fc6cb378cbbedd83c28b7b0"}
Jul 13 21:11:47.381619 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.381578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:47.382937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.382908    2577 generic.go:356] "Generic (PLEG): container finished" podID="f79973c8-85c1-4284-b467-3788edeb087a" containerID="c606ff19764a9484a6c5072a242b62713c17d3b94581d560abe8a5e8dac6cdd2" exitCode=0
Jul 13 21:11:47.383083 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.382959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerDied","Data":"c606ff19764a9484a6c5072a242b62713c17d3b94581d560abe8a5e8dac6cdd2"}
Jul 13 21:11:47.431707 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.431673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:47.966614 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.966584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:47.966753 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:47.966727    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:11:47.966843 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:47.966754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:47.966843 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:47.966798    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:51.966782856 +0000 UTC m=+43.383709179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:11:47.966956 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:47.966852    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:11:47.966956 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:47.966868    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:11:47.966956 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:47.966913    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:11:51.966901358 +0000 UTC m=+43.383827684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:11:48.387030 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.386995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8w765" event={"ID":"edb8e8a1-8d39-4f54-9858-158909fbd6fb","Type":"ContainerStarted","Data":"4a516710d037340c6d5fdafab4e288afa4e38196190fe47a5eb486564f67a886"}
Jul 13 21:11:48.390234 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.390205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9msh7" event={"ID":"f79973c8-85c1-4284-b467-3788edeb087a","Type":"ContainerStarted","Data":"7a08c2bfa94e431a65e9e77421630f5aa754d9638ae5c69c11e91bdec84dc9b0"}
Jul 13 21:11:48.402153 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.402111    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8w765" podStartSLOduration=2.206824664 podStartE2EDuration="4.402097046s" totalImagesPullingTime="2.195272382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:48.40141904 +0000 UTC m=+39.818345385" watchObservedRunningTime="2026-07-13 21:11:48.402097046 +0000 UTC m=+39.819023367"
Jul 13 21:11:48.424038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.423986    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9msh7" podStartSLOduration=9.50498679 podStartE2EDuration="39.423970307s" totalImagesPullingTime="29.918983517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:48.423367222 +0000 UTC m=+39.840293566" watchObservedRunningTime="2026-07-13 21:11:48.423970307 +0000 UTC m=+39.840896651"
Jul 13 21:11:48.547492 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.547459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:48.643432 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.643354    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:48.958016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:48.957942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:49.019052 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:49.019024    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:49.959237 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:49.959204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:50.016443 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:50.016413    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:51.706395 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.706361    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qcwbl"]
Jul 13 21:11:51.727742 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.727705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:51.733109 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.733090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qcwbl"]
Jul 13 21:11:51.733207 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.733196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.736057 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.736033    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 21:11:51.736578 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.736552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 21:11:51.736700 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.736556    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 21:11:51.736700 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.736558    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hmn24\""
Jul 13 21:11:51.736700 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.736562    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 21:11:51.894895 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.894860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c69d4585-bbb0-424c-a341-dc2ac1987cc4-crio-socket\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.895076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.894911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzwlc\" (UniqueName: \"kubernetes.io/projected/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-api-access-kzwlc\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.895076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.894981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.895076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.895041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c69d4585-bbb0-424c-a341-dc2ac1987cc4-data-volume\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.895076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.895064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.982687 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.982596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:51.995809 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:11:51.995881 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.995881 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995847    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c69d4585-bbb0-424c-a341-dc2ac1987cc4-data-volume\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.995881 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.995974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:11:51.995974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995951    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c69d4585-bbb0-424c-a341-dc2ac1987cc4-crio-socket\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.996057 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.995996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzwlc\" (UniqueName: \"kubernetes.io/projected/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-api-access-kzwlc\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.996057 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996008    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:51.996138 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996063    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:11:51.996138 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996082    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:11:51.996138 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996103    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:11:51.996138 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996070    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:52.496051637 +0000 UTC m=+43.912977966 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:51.996138 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996140    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:11:59.996124292 +0000 UTC m=+51.413050615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:11:51.996355 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996155    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:52.496146423 +0000 UTC m=+43.913072751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : configmap "kube-rbac-proxy" not found
Jul 13 21:11:51.996355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.996154    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c69d4585-bbb0-424c-a341-dc2ac1987cc4-data-volume\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.996355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:51.996199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c69d4585-bbb0-424c-a341-dc2ac1987cc4-crio-socket\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:51.996355 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996206    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:11:51.996355 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:51.996293    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:59.996274023 +0000 UTC m=+51.413200345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:11:52.005782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.005737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzwlc\" (UniqueName: \"kubernetes.io/projected/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-api-access-kzwlc\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:52.086601 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.086571    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.329982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.329900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.384725 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.384697    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.499972 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.499939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:52.500122 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.499983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:52.500122 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:52.500075    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:11:52.500211 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:52.500135    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:53.500120047 +0000 UTC m=+44.917046369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : configmap "kube-rbac-proxy" not found
Jul 13 21:11:52.500211 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:52.500184    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:52.500300 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:52.500229    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:53.500218764 +0000 UTC m=+44.917145085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:52.639058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.639025    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.702816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.702784    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"]
Jul 13 21:11:52.722810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.722785    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.725229 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.725212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"]
Jul 13 21:11:52.725316 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.725305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"
Jul 13 21:11:52.727780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.727745    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 13 21:11:52.727912 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.727833    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 13 21:11:52.728857 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.728838    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zx9xn\""
Jul 13 21:11:52.785156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.785121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:52.902932 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:52.902839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2zmd\" (UniqueName: \"kubernetes.io/projected/0d554ace-4ba7-438f-a4e0-c14b23e81b70-kube-api-access-q2zmd\") pod \"migrator-7f59b54f6c-2bm8d\" (UID: \"0d554ace-4ba7-438f-a4e0-c14b23e81b70\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"
Jul 13 21:11:53.003796 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.003738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2zmd\" (UniqueName: \"kubernetes.io/projected/0d554ace-4ba7-438f-a4e0-c14b23e81b70-kube-api-access-q2zmd\") pod \"migrator-7f59b54f6c-2bm8d\" (UID: \"0d554ace-4ba7-438f-a4e0-c14b23e81b70\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"
Jul 13 21:11:53.011192 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.011161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2zmd\" (UniqueName: \"kubernetes.io/projected/0d554ace-4ba7-438f-a4e0-c14b23e81b70-kube-api-access-q2zmd\") pod \"migrator-7f59b54f6c-2bm8d\" (UID: \"0d554ace-4ba7-438f-a4e0-c14b23e81b70\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"
Jul 13 21:11:53.034275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.034237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"
Jul 13 21:11:53.180935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.180914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d"]
Jul 13 21:11:53.183231 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:53.183202    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d554ace_4ba7_438f_a4e0_c14b23e81b70.slice/crio-1454b0c6ae2e176f9a32f4df806594434ae7761dc5165120d7412b5d1299dd0b WatchSource:0}: Error finding container 1454b0c6ae2e176f9a32f4df806594434ae7761dc5165120d7412b5d1299dd0b: Status 404 returned error can't find the container with id 1454b0c6ae2e176f9a32f4df806594434ae7761dc5165120d7412b5d1299dd0b
Jul 13 21:11:53.403218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.403172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d" event={"ID":"0d554ace-4ba7-438f-a4e0-c14b23e81b70","Type":"ContainerStarted","Data":"1454b0c6ae2e176f9a32f4df806594434ae7761dc5165120d7412b5d1299dd0b"}
Jul 13 21:11:53.480792 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.480704    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zngbn_0729b1c1-8389-4d40-ab63-87e52c39942e/dns-node-resolver/0.log"
Jul 13 21:11:53.508314 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.508279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:53.508439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:53.508326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:53.508439 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:53.508402    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:11:53.508544 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:53.508473    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:53.508544 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:53.508486    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:55.508469029 +0000 UTC m=+46.925395369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : configmap "kube-rbac-proxy" not found
Jul 13 21:11:53.508544 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:53.508538    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:55.508521246 +0000 UTC m=+46.925447573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:54.079612 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.079582    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98x85_23115f88-178a-4e5b-9c3f-34a160fb24e3/node-ca/0.log"
Jul 13 21:11:54.202455 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.202422    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hwpmp"]
Jul 13 21:11:54.215090 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.215056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hwpmp"]
Jul 13 21:11:54.215252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.215165    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.217726 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.217701    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 13 21:11:54.217726 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.217721    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 13 21:11:54.218715 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.218521    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zww9c\""
Jul 13 21:11:54.218715 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.218570    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 13 21:11:54.219014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.218999    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 13 21:11:54.238826 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.238792    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:54.315596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.315563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-cabundle\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.315823 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.315653    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-key\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.315823 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.315688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nxnw\" (UniqueName: \"kubernetes.io/projected/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-kube-api-access-6nxnw\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.416661 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.416628    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-key\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.416661 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.416660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nxnw\" (UniqueName: \"kubernetes.io/projected/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-kube-api-access-6nxnw\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.416896 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.416844    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-cabundle\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.417477 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.417451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-cabundle\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.418983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.418966    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-signing-key\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.424410 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.424388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nxnw\" (UniqueName: \"kubernetes.io/projected/ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d-kube-api-access-6nxnw\") pod \"service-ca-79f68b9f4f-hwpmp\" (UID: \"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.525744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.525709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp"
Jul 13 21:11:54.552389 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.552353    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:54.816099 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.816067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hwpmp"]
Jul 13 21:11:54.819033 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:11:54.819003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec3706fc_eef4_4e70_9c35_0c0d7cdfaa5d.slice/crio-b7ec867acf6e601ae571b533e38c075245840c9ece65fd727afd78e2acd3dae7 WatchSource:0}: Error finding container b7ec867acf6e601ae571b533e38c075245840c9ece65fd727afd78e2acd3dae7: Status 404 returned error can't find the container with id b7ec867acf6e601ae571b533e38c075245840c9ece65fd727afd78e2acd3dae7
Jul 13 21:11:54.880780 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:54.880736    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8w765_edb8e8a1-8d39-4f54-9858-158909fbd6fb/serve-healthcheck-canary/0.log"
Jul 13 21:11:55.087036 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.086996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:55.411724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.411687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d" event={"ID":"0d554ace-4ba7-438f-a4e0-c14b23e81b70","Type":"ContainerStarted","Data":"4b6c08249d3c529d14c1797374c989c29e5fdcde981ac1ca5bedda13b1a8d380"}
Jul 13 21:11:55.411948 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.411778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d" event={"ID":"0d554ace-4ba7-438f-a4e0-c14b23e81b70","Type":"ContainerStarted","Data":"379eefde21bddd4ba6130070201f9a63a3f4bedf05efa8cb16fd5e73b4b82f3f"}
Jul 13 21:11:55.412978 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.412955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp" event={"ID":"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d","Type":"ContainerStarted","Data":"b7ec867acf6e601ae571b533e38c075245840c9ece65fd727afd78e2acd3dae7"}
Jul 13 21:11:55.449296 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.449236    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-2bm8d" podStartSLOduration=1.9488855539999999 podStartE2EDuration="3.449218726s" totalImagesPullingTime="1.500333172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.435084908 +0000 UTC m=+46.852011251" watchObservedRunningTime="2026-07-13 21:11:55.449218726 +0000 UTC m=+46.866145070"
Jul 13 21:11:55.525280 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.525250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:55.525382 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:55.525293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:55.525420 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:55.525371    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 13 21:11:55.525454 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:55.525423    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:55.525454 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:55.525439    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:59.525423445 +0000 UTC m=+50.942349767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : configmap "kube-rbac-proxy" not found
Jul 13 21:11:55.525521 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:55.525466    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:59.525451791 +0000 UTC m=+50.942378113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:56.578062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:56.578031    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:57.778169 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:57.778138    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:58.422744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:58.422704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp" event={"ID":"ec3706fc-eef4-4e70-9c35-0c0d7cdfaa5d","Type":"ContainerStarted","Data":"7417aaf86487e3f891862b9ad0c622285e02bd8f51555e36fcb6cf433beb2946"}
Jul 13 21:11:58.445568 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:58.445517    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-hwpmp" podStartSLOduration=1.437400109 podStartE2EDuration="4.445497266s" totalImagesPullingTime="3.008097157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:58.445202602 +0000 UTC m=+49.862128952" watchObservedRunningTime="2026-07-13 21:11:58.445497266 +0000 UTC m=+49.862423610"
Jul 13 21:11:59.068314 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.068281    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:59.452032 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.451996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:59.533698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.533650    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:59.559444 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.559408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:59.559668 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:59.559635    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:59.559969 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:11:59.559955    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls podName:c69d4585-bbb0-424c-a341-dc2ac1987cc4 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:07.559932666 +0000 UTC m=+58.976858996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qcwbl" (UID: "c69d4585-bbb0-424c-a341-dc2ac1987cc4") : secret "insights-runtime-extractor-tls" not found
Jul 13 21:11:59.560124 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.559835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:59.560542 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.560523    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c69d4585-bbb0-424c-a341-dc2ac1987cc4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:11:59.634079 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.634045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:59.690826 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.690787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:11:59.817439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:11:59.817362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:00.064526 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:00.064490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") pod \"image-registry-69bffb78b7-r2865\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") " pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:12:00.064718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:00.064545    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:00.064806 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:00.064716    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 13 21:12:00.064806 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:00.064735    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69bffb78b7-r2865: secret "image-registry-tls" not found
Jul 13 21:12:00.064806 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:00.064773    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 13 21:12:00.064956 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:00.064821    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls podName:abf95581-be7a-48ff-a634-9b3a4579c55a nodeName:}" failed. No retries permitted until 2026-07-13 21:12:16.064801069 +0000 UTC m=+67.481727409 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls") pod "image-registry-69bffb78b7-r2865" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a") : secret "image-registry-tls" not found
Jul 13 21:12:00.064956 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:00.064842    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls podName:7c927257-e779-4148-a577-c9bec9ad63f2 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:16.064831679 +0000 UTC m=+67.481758026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls") pod "dns-default-kqxx5" (UID: "7c927257-e779-4148-a577-c9bec9ad63f2") : secret "dns-default-metrics-tls" not found
Jul 13 21:12:00.820775 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:00.820741    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:01.007617 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:01.007573    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:01.125107 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:01.125022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:01.422649 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:01.422608    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:01.615131 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:01.615096    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:02.212517 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:02.212466    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:02.617090 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:02.617003    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:02.764241 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:02.764207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:02.928220 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:02.928184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:03.028594 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:03.028553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:03.411346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:03.411317    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:03.609567 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:03.609536    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:03.658088 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:03.658056    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:04.207488 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:04.207453    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:04.609497 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:04.609418    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:04.944719 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:04.944688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:05.047435 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:05.047401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:05.146812 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:05.146783    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:05.250845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:05.250754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:05.367958 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:05.367925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:05.812267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:05.812233    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:06.206715 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:06.206680    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:06.613286 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:06.613205    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:07.009803 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.009749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:07.410281 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.410234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:07.630115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.630060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:12:07.632446 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.632408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c69d4585-bbb0-424c-a341-dc2ac1987cc4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qcwbl\" (UID: \"c69d4585-bbb0-424c-a341-dc2ac1987cc4\") " pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:12:07.642329 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.642304    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qcwbl"
Jul 13 21:12:07.771450 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.769736    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qcwbl"]
Jul 13 21:12:07.773566 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:07.773533    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69d4585_bbb0_424c_a341_dc2ac1987cc4.slice/crio-54815ea36cd6c5f25d84f4e637da75e8a68beacee29a264f5558b5613caf734e WatchSource:0}: Error finding container 54815ea36cd6c5f25d84f4e637da75e8a68beacee29a264f5558b5613caf734e: Status 404 returned error can't find the container with id 54815ea36cd6c5f25d84f4e637da75e8a68beacee29a264f5558b5613caf734e
Jul 13 21:12:07.812850 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:07.812811    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:08.010720 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:08.010631    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:08.409137 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:08.409065    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:08.451796 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:08.451740    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qcwbl" event={"ID":"c69d4585-bbb0-424c-a341-dc2ac1987cc4","Type":"ContainerStarted","Data":"54815ea36cd6c5f25d84f4e637da75e8a68beacee29a264f5558b5613caf734e"}
Jul 13 21:12:09.006114 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:09.006076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:09.415408 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:09.415368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:09.610284 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:09.610225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:09.858091 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:09.858012    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:10.370321 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:10.370288    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v6skj"
Jul 13 21:12:12.466488 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.466446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qcwbl" event={"ID":"c69d4585-bbb0-424c-a341-dc2ac1987cc4","Type":"ContainerStarted","Data":"c17bea2a35791370a3acf8c9139357d5ca1487cd8dad4d862d9a326d69a522ff"}
Jul 13 21:12:12.753066 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.752948    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:12.865242 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.865209    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:12.949827 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.949790    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"]
Jul 13 21:12:12.951970 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.951946    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:12.956056 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.956031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hm7pv\""
Jul 13 21:12:12.956191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.956031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 13 21:12:12.956191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.956081    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 13 21:12:12.956191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.956041    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 13 21:12:12.956191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.956179    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 13 21:12:12.972970 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.972944    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"]
Jul 13 21:12:12.974980 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.974960    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:12.977664 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.977625    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 13 21:12:12.977935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.977918    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 13 21:12:12.978880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.978858    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 13 21:12:12.978880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.978877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 13 21:12:12.985885 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:12.985848    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"]
Jul 13 21:12:13.001423 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.001391    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"]
Jul 13 21:12:13.021390 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.021306    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69bffb78b7-r2865"]
Jul 13 21:12:13.021584 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:13.021558    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-69bffb78b7-r2865" podUID="abf95581-be7a-48ff-a634-9b3a4579c55a"
Jul 13 21:12:13.029138 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.029099    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.082269 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50658322-e331-42c1-bfa6-b86f79b3b62b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.082465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.082465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqjc9\" (UniqueName: \"kubernetes.io/projected/50658322-e331-42c1-bfa6-b86f79b3b62b-kube-api-access-nqjc9\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.082465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.082638 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f487afc-1904-49ce-9d74-e739c0619c54-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.082638 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082627    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.082738 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.082738 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.082707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2jn\" (UniqueName: \"kubernetes.io/projected/6f487afc-1904-49ce-9d74-e739c0619c54-kube-api-access-6h2jn\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.122668 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.122618    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.125793 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.125750    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-567dfb87dc-cj76z"]
Jul 13 21:12:13.129859 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.128649    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.143751 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.143721    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-567dfb87dc-cj76z"]
Jul 13 21:12:13.183622 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-trusted-ca\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.183828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5mnq\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-kube-api-access-p5mnq\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.183828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-tls\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.183828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-certificates\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.183828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-installation-pull-secrets\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.184012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adac3e94-350e-48e3-a19b-fa521ffb5884-ca-trust-extracted\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.184012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183924    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.183960    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2jn\" (UniqueName: \"kubernetes.io/projected/6f487afc-1904-49ce-9d74-e739c0619c54-kube-api-access-6h2jn\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50658322-e331-42c1-bfa6-b86f79b3b62b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqjc9\" (UniqueName: \"kubernetes.io/projected/50658322-e331-42c1-bfa6-b86f79b3b62b-kube-api-access-nqjc9\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184112    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-image-registry-private-configuration\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184140    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-bound-sa-token\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f487afc-1904-49ce-9d74-e739c0619c54-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.184967 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.184921    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f487afc-1904-49ce-9d74-e739c0619c54-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.187462 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.187222    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-ca\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.187462 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.187279    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.187462 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.187373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.187834 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.187750    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/50658322-e331-42c1-bfa6-b86f79b3b62b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.188410 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.188316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/6f487afc-1904-49ce-9d74-e739c0619c54-hub\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.197805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.197748    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqjc9\" (UniqueName: \"kubernetes.io/projected/50658322-e331-42c1-bfa6-b86f79b3b62b-kube-api-access-nqjc9\") pod \"managed-serviceaccount-addon-agent-789dffc59b-k6r7q\" (UID: \"50658322-e331-42c1-bfa6-b86f79b3b62b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.198003 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.197978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2jn\" (UniqueName: \"kubernetes.io/projected/6f487afc-1904-49ce-9d74-e739c0619c54-kube-api-access-6h2jn\") pod \"cluster-proxy-proxy-agent-f6587bc5-5zwtt\" (UID: \"6f487afc-1904-49ce-9d74-e739c0619c54\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.218105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.218073    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.277613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.277526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"
Jul 13 21:12:13.285538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285510    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-trusted-ca\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.285704 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5mnq\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-kube-api-access-p5mnq\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.285839 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-tls\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.285839 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-certificates\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.285975 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-installation-pull-secrets\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.285975 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.285917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adac3e94-350e-48e3-a19b-fa521ffb5884-ca-trust-extracted\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.286087 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.286015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-image-registry-private-configuration\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.286087 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.286040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-bound-sa-token\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.286895 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.286850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-trusted-ca\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.287017 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.286997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"
Jul 13 21:12:13.287236 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.287189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/adac3e94-350e-48e3-a19b-fa521ffb5884-ca-trust-extracted\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.287236 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.287219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-certificates\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.289253 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.289205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-image-registry-private-configuration\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.289361 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.289314    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-registry-tls\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.289519 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.289494    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/adac3e94-350e-48e3-a19b-fa521ffb5884-installation-pull-secrets\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.296991 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.296956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-bound-sa-token\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.297395 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.297355    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5mnq\" (UniqueName: \"kubernetes.io/projected/adac3e94-350e-48e3-a19b-fa521ffb5884-kube-api-access-p5mnq\") pod \"image-registry-567dfb87dc-cj76z\" (UID: \"adac3e94-350e-48e3-a19b-fa521ffb5884\") " pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.305797 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.305744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.386870 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.386782    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.435136 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.435106    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q"]
Jul 13 21:12:13.438816 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:13.438779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod50658322_e331_42c1_bfa6_b86f79b3b62b.slice/crio-c71ba41c33a9bd67fbb2ebc8333690c561420e1f8d9a5ff872bb257ee0e7ac39 WatchSource:0}: Error finding container c71ba41c33a9bd67fbb2ebc8333690c561420e1f8d9a5ff872bb257ee0e7ac39: Status 404 returned error can't find the container with id c71ba41c33a9bd67fbb2ebc8333690c561420e1f8d9a5ff872bb257ee0e7ac39
Jul 13 21:12:13.452312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.452283    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2ksct\""
Jul 13 21:12:13.456473 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.456447    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt"]
Jul 13 21:12:13.459736 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:13.459704    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f487afc_1904_49ce_9d74_e739c0619c54.slice/crio-ce3cb04bb096ff2a3e69160297ddaedab711f4dfe0669b9fbee063ce2bccb585 WatchSource:0}: Error finding container ce3cb04bb096ff2a3e69160297ddaedab711f4dfe0669b9fbee063ce2bccb585: Status 404 returned error can't find the container with id ce3cb04bb096ff2a3e69160297ddaedab711f4dfe0669b9fbee063ce2bccb585
Jul 13 21:12:13.460587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.460565    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:13.466232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.466204    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.471153 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.471127    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt" event={"ID":"6f487afc-1904-49ce-9d74-e739c0619c54","Type":"ContainerStarted","Data":"ce3cb04bb096ff2a3e69160297ddaedab711f4dfe0669b9fbee063ce2bccb585"}
Jul 13 21:12:13.472664 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.472625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q" event={"ID":"50658322-e331-42c1-bfa6-b86f79b3b62b","Type":"ContainerStarted","Data":"c71ba41c33a9bd67fbb2ebc8333690c561420e1f8d9a5ff872bb257ee0e7ac39"}
Jul 13 21:12:13.474962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.474885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qcwbl" event={"ID":"c69d4585-bbb0-424c-a341-dc2ac1987cc4","Type":"ContainerStarted","Data":"78414f43d5ff3bbcb25837fee843b226d5ddcd77568f8126980d4a79c657a9ae"}
Jul 13 21:12:13.474962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.474928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:12:13.482251 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.482228    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:12:13.588005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.587950    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.587993    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588238 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588238 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588238 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588220    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:12:13.588386 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588362    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588446 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588401    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2vlr\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588499 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588456    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token\") pod \"abf95581-be7a-48ff-a634-9b3a4579c55a\" (UID: \"abf95581-be7a-48ff-a634-9b3a4579c55a\") "
Jul 13 21:12:13.588603 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588576    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:12:13.588737 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588715    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:12:13.588859 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588844    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/abf95581-be7a-48ff-a634-9b3a4579c55a-ca-trust-extracted\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.588965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588863    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-certificates\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.588965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.588882    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/abf95581-be7a-48ff-a634-9b3a4579c55a-trusted-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.591546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.591504    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:12:13.591684 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.591598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr" (OuterVolumeSpecName: "kube-api-access-s2vlr") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "kube-api-access-s2vlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:12:13.591684 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.591624    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:12:13.591846 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.591823    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "abf95581-be7a-48ff-a634-9b3a4579c55a" (UID: "abf95581-be7a-48ff-a634-9b3a4579c55a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:12:13.605854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.605824    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-567dfb87dc-cj76z"]
Jul 13 21:12:13.609289 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:13.609262    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadac3e94_350e_48e3_a19b_fa521ffb5884.slice/crio-22ec439bfc4030e0944815bafd8d81668206d8021438baf003ca7801873d0487 WatchSource:0}: Error finding container 22ec439bfc4030e0944815bafd8d81668206d8021438baf003ca7801873d0487: Status 404 returned error can't find the container with id 22ec439bfc4030e0944815bafd8d81668206d8021438baf003ca7801873d0487
Jul 13 21:12:13.690216 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.690186    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-image-registry-private-configuration\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.690216 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.690212    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/abf95581-be7a-48ff-a634-9b3a4579c55a-installation-pull-secrets\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.690216 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.690222    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2vlr\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-kube-api-access-s2vlr\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.690435 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.690232    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-bound-sa-token\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:13.815675 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.815588    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:13.916012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:13.915980    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.016412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.016375    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.081893 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.081799    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.398624 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.398590    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.480256 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.480226    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69bffb78b7-r2865"
Jul 13 21:12:14.480256 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.480227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" event={"ID":"adac3e94-350e-48e3-a19b-fa521ffb5884","Type":"ContainerStarted","Data":"8d97cbf5c794faa3f81c1ec5df4f574595389e83426cb1e5f530d7226f8eff61"}
Jul 13 21:12:14.480716 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.480284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" event={"ID":"adac3e94-350e-48e3-a19b-fa521ffb5884","Type":"ContainerStarted","Data":"22ec439bfc4030e0944815bafd8d81668206d8021438baf003ca7801873d0487"}
Jul 13 21:12:14.496021 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.495992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.501842 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.501795    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" podStartSLOduration=1.501740082 podStartE2EDuration="1.501740082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:14.501020251 +0000 UTC m=+65.917946611" watchObservedRunningTime="2026-07-13 21:12:14.501740082 +0000 UTC m=+65.918666428"
Jul 13 21:12:14.531924 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.531894    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69bffb78b7-r2865"]
Jul 13 21:12:14.538941 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.538908    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69bffb78b7-r2865"]
Jul 13 21:12:14.599108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.599066    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/abf95581-be7a-48ff-a634-9b3a4579c55a-registry-tls\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:12:14.635332 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.635296    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.857869 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.857709    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:14.901100 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.901063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:12:14.901281 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.901124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:12:14.903589 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.903566    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:12:14.903728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.903591    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:12:14.914244 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.914210    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7476560c-3a7d-42bc-88a6-7f2492d06a09-original-pull-secret\") pod \"global-pull-secret-syncer-bjjbg\" (UID: \"7476560c-3a7d-42bc-88a6-7f2492d06a09\") " pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:12:14.914399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.914317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab-metrics-certs\") pod \"network-metrics-daemon-rzds7\" (UID: \"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab\") " pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:12:14.953925 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:14.953880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:15.100134 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.100106    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vxg2k\""
Jul 13 21:12:15.103115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.103093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:12:15.105261 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.105247    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:12:15.109020 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.108968    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rzds7"
Jul 13 21:12:15.109020 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.109001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bjjbg"
Jul 13 21:12:15.115551 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.115526    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:12:15.126376 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.126350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gxvp\" (UniqueName: \"kubernetes.io/projected/54094210-e8fb-4c14-95da-7ab6dd6d35a5-kube-api-access-8gxvp\") pod \"network-check-target-27v2r\" (UID: \"54094210-e8fb-4c14-95da-7ab6dd6d35a5\") " pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:12:15.186665 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.186631    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf95581-be7a-48ff-a634-9b3a4579c55a" path="/var/lib/kubelet/pods/abf95581-be7a-48ff-a634-9b3a4579c55a/volumes"
Jul 13 21:12:15.274708 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.274672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rzds7"]
Jul 13 21:12:15.297728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.297648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bjjbg"]
Jul 13 21:12:15.392490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.392461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t2zb8\""
Jul 13 21:12:15.400437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.400401    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:12:15.490662 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.490623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qcwbl" event={"ID":"c69d4585-bbb0-424c-a341-dc2ac1987cc4","Type":"ContainerStarted","Data":"3f5e568f8956ee6804129bfb0eea3db1d19b5c161db56a116a914483e7fe4e60"}
Jul 13 21:12:15.492905 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.492878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzds7" event={"ID":"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab","Type":"ContainerStarted","Data":"ad4a76a31860cde9493508ae721418f645e5df6f220fa8aa2632c148477dfe48"}
Jul 13 21:12:15.495573 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.495540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjjbg" event={"ID":"7476560c-3a7d-42bc-88a6-7f2492d06a09","Type":"ContainerStarted","Data":"d8b0a1e38190fe2ba810c9d9fbb7861d01ee17eadd960502490788d7ba967381"}
Jul 13 21:12:15.495695 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.495601    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:15.509804 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.509603    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qcwbl" podStartSLOduration=17.304611768 podStartE2EDuration="24.509587284s" totalImagesPullingTime="7.204975516s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:15.508627058 +0000 UTC m=+66.925553416" watchObservedRunningTime="2026-07-13 21:12:15.509587284 +0000 UTC m=+66.926513683"
Jul 13 21:12:15.580541 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.580460    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27v2r"]
Jul 13 21:12:15.584189 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:15.584139    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54094210_e8fb_4c14_95da_7ab6dd6d35a5.slice/crio-2719d7cf25ccf9f94175c38e48d3b6321d30c1da69a29c4d4345f76b4d62fcab WatchSource:0}: Error finding container 2719d7cf25ccf9f94175c38e48d3b6321d30c1da69a29c4d4345f76b4d62fcab: Status 404 returned error can't find the container with id 2719d7cf25ccf9f94175c38e48d3b6321d30c1da69a29c4d4345f76b4d62fcab
Jul 13 21:12:15.852207 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:15.852089    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:16.111112 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:16.111027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:16.115736 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:16.115696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7c927257-e779-4148-a577-c9bec9ad63f2-metrics-tls\") pod \"dns-default-kqxx5\" (UID: \"7c927257-e779-4148-a577-c9bec9ad63f2\") " pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:16.327531 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:16.327255    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-btz4g\""
Jul 13 21:12:16.335319 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:16.335285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:16.503422 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:16.503335    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27v2r" event={"ID":"54094210-e8fb-4c14-95da-7ab6dd6d35a5","Type":"ContainerStarted","Data":"2719d7cf25ccf9f94175c38e48d3b6321d30c1da69a29c4d4345f76b4d62fcab"}
Jul 13 21:12:18.446149 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:18.446108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:18.501248 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:18.501219    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:19.944242 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:19.944207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:20.002259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:20.002223    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:21.248377 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:21.248339    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:21.299484 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:21.299448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:22.190553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.190076    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kqxx5"]
Jul 13 21:12:22.194208 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:22.194147    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c927257_e779_4148_a577_c9bec9ad63f2.slice/crio-e35df968930848aab25282cc92623b563a5ab405f44abc869c1174a454b99b25 WatchSource:0}: Error finding container e35df968930848aab25282cc92623b563a5ab405f44abc869c1174a454b99b25: Status 404 returned error can't find the container with id e35df968930848aab25282cc92623b563a5ab405f44abc869c1174a454b99b25
Jul 13 21:12:22.529141 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.529045    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzds7" event={"ID":"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab","Type":"ContainerStarted","Data":"94168c503d7389229190f7e39d9fce73755c6a2d0d543706b806d8f5d18749c1"}
Jul 13 21:12:22.529141 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.529121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rzds7" event={"ID":"fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab","Type":"ContainerStarted","Data":"d206c45026a916bd0f3e15a3cd25b1d2af8df231c51acc8e0ba37b26d42e8d49"}
Jul 13 21:12:22.530513 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.530481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kqxx5" event={"ID":"7c927257-e779-4148-a577-c9bec9ad63f2","Type":"ContainerStarted","Data":"e35df968930848aab25282cc92623b563a5ab405f44abc869c1174a454b99b25"}
Jul 13 21:12:22.531997 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.531970    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bjjbg" event={"ID":"7476560c-3a7d-42bc-88a6-7f2492d06a09","Type":"ContainerStarted","Data":"7a6fdce4733c3b32674b287eeafb9eefbddae3a791a26265ff947fba89aa2733"}
Jul 13 21:12:22.534455 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.534424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt" event={"ID":"6f487afc-1904-49ce-9d74-e739c0619c54","Type":"ContainerStarted","Data":"4dc3022cdd4e7b096dc60fb98701f5565f9534284ff0dcf509ef8a4e28793302"}
Jul 13 21:12:22.534566 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.534460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt" event={"ID":"6f487afc-1904-49ce-9d74-e739c0619c54","Type":"ContainerStarted","Data":"2a8830e4d5b496f9964ff48943b39a36c7d2e3cdce83a56bb9975f4c6a10776a"}
Jul 13 21:12:22.534566 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.534476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt" event={"ID":"6f487afc-1904-49ce-9d74-e739c0619c54","Type":"ContainerStarted","Data":"48398029f8b3a359943f6b442139f5bc75939291cb4adfd058bf50f0365bd36e"}
Jul 13 21:12:22.536239 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.536213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q" event={"ID":"50658322-e331-42c1-bfa6-b86f79b3b62b","Type":"ContainerStarted","Data":"561884b0c148861929f98f25120ab370c378f6056291d18c75b8311eaf9ce8d6"}
Jul 13 21:12:22.537717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.537687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27v2r" event={"ID":"54094210-e8fb-4c14-95da-7ab6dd6d35a5","Type":"ContainerStarted","Data":"1078806d4da50e49e5e83aa08c5e76276060f161ea738422c4de5327cadbd0c3"}
Jul 13 21:12:22.537888 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.537875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:12:22.544873 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.544834    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rzds7" podStartSLOduration=66.805109841 podStartE2EDuration="1m13.54482198s" totalImagesPullingTime="6.739712139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:22.543827489 +0000 UTC m=+73.960753830" watchObservedRunningTime="2026-07-13 21:12:22.54482198 +0000 UTC m=+73.961748321"
Jul 13 21:12:22.561025 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.560958    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6587bc5-5zwtt" podStartSLOduration=1.980265088 podStartE2EDuration="10.560937075s" totalImagesPullingTime="8.580671987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:22.560689523 +0000 UTC m=+73.977615895" watchObservedRunningTime="2026-07-13 21:12:22.560937075 +0000 UTC m=+73.977863419"
Jul 13 21:12:22.574904 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.574833    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-27v2r" podStartSLOduration=67.121851605 podStartE2EDuration="1m13.574814656s" totalImagesPullingTime="6.452963051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:22.57446538 +0000 UTC m=+73.991391721" watchObservedRunningTime="2026-07-13 21:12:22.574814656 +0000 UTC m=+73.991741001"
Jul 13 21:12:22.588502 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.588379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-789dffc59b-k6r7q" podStartSLOduration=2.002475808 podStartE2EDuration="10.588362127s" totalImagesPullingTime="8.585886319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:22.587912765 +0000 UTC m=+74.004839112" watchObservedRunningTime="2026-07-13 21:12:22.588362127 +0000 UTC m=+74.005288472"
Jul 13 21:12:22.602959 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.602908    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bjjbg" podStartSLOduration=66.86439195 podStartE2EDuration="1m13.602893932s" totalImagesPullingTime="6.738501982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:22.601805524 +0000 UTC m=+74.018731867" watchObservedRunningTime="2026-07-13 21:12:22.602893932 +0000 UTC m=+74.019820275"
Jul 13 21:12:22.665982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.665954    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:22.765990 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:22.765937    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:23.250183 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:23.250150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:23.301480 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:23.301444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:24.547367 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.547331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kqxx5" event={"ID":"7c927257-e779-4148-a577-c9bec9ad63f2","Type":"ContainerStarted","Data":"3793c0c1b293c21cf29149d73acdab7ee9c3510afe195a1b27d18e29ed61827f"}
Jul 13 21:12:24.547814 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.547383    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kqxx5" event={"ID":"7c927257-e779-4148-a577-c9bec9ad63f2","Type":"ContainerStarted","Data":"edd80778cadedc2dec101ca002c20f7e562716a875c1abcf07e2153558750f26"}
Jul 13 21:12:24.547814 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.547463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:24.563957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.563906    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kqxx5" podStartSLOduration=38.941469904 podStartE2EDuration="40.563888738s" totalImagesPullingTime="1.622418834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:24.562539116 +0000 UTC m=+75.979465464" watchObservedRunningTime="2026-07-13 21:12:24.563888738 +0000 UTC m=+75.980815082"
Jul 13 21:12:24.644601 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.644569    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:24.745857 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.745825    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:24.824755 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:24.824664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:25.069156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:25.069124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:25.599470 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:25.599427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:25.655434 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:25.655405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:27.245016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:27.244977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:27.345872 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:27.345840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:28.645176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:28.645144    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:29.856877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:29.856824    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:29.920357 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:29.920280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:29.985587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:29.985561    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:32.237259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:32.237230    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:33.465056 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:33.465020    2577 patch_prober.go:28] interesting pod/image-registry-567dfb87dc-cj76z container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 21:12:33.465425 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:33.465073    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" podUID="adac3e94-350e-48e3-a19b-fa521ffb5884" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:12:33.474593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:33.474565    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:33.575334 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:33.575306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:33.676266 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:33.676235    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.011160 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.011127    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.204259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.204222    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.291040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.290952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.296586 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.296557    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c5pcj"]
Jul 13 21:12:34.300038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.300019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.302845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.302685    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 21:12:34.302845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.302716    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 21:12:34.302845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.302801    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 21:12:34.302845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.302725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 21:12:34.304018 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.303988    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nphg5\""
Jul 13 21:12:34.304123 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.304031    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 21:12:34.304123 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.304036    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 21:12:34.357311 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.357279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.452867 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.452827    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.452893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-textfile\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.452930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk8tb\" (UniqueName: \"kubernetes.io/projected/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-kube-api-access-pk8tb\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.452955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-wtmp\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.452979    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-metrics-client-ca\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453171 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.453037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-sys\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453171 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.453085    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453171 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.453102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.453171 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.453121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-root\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.500334 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.500292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.554560 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-textfile\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554560 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk8tb\" (UniqueName: \"kubernetes.io/projected/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-kube-api-access-pk8tb\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554560 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-wtmp\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-metrics-client-ca\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-sys\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554695    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-root\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-wtmp\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.554880 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:34.554868    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 13 21:12:34.555290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-textfile\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.555290 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:12:34.554943    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls podName:ada5e595-ce6b-4486-8a21-60e7a7ffd4c4 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:35.054921569 +0000 UTC m=+86.471847895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls") pod "node-exporter-c5pcj" (UID: "ada5e595-ce6b-4486-8a21-60e7a7ffd4c4") : secret "node-exporter-tls" not found
Jul 13 21:12:34.555290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554953    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-root\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.555290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.554998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-sys\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.555519 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.555497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-metrics-client-ca\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.555572 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.555557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.556033 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.556014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kqxx5"
Jul 13 21:12:34.557195 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.557173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.568034 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.568009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk8tb\" (UniqueName: \"kubernetes.io/projected/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-kube-api-access-pk8tb\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:34.609191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.609156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.710756 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.710716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:34.792303 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:34.792264    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:35.059081 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.059036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:35.061340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.061321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ada5e595-ce6b-4486-8a21-60e7a7ffd4c4-node-exporter-tls\") pod \"node-exporter-c5pcj\" (UID: \"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4\") " pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:35.210439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.210404    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c5pcj"
Jul 13 21:12:35.220114 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:12:35.220085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada5e595_ce6b_4486_8a21_60e7a7ffd4c4.slice/crio-4320531b8d8bb2fd61352c29c688400131c47bf1995b604feac331a8404f644d WatchSource:0}: Error finding container 4320531b8d8bb2fd61352c29c688400131c47bf1995b604feac331a8404f644d: Status 404 returned error can't find the container with id 4320531b8d8bb2fd61352c29c688400131c47bf1995b604feac331a8404f644d
Jul 13 21:12:35.235461 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.235434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:35.339474 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.339390    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:35.551038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.551006    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:35.592804 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:35.592701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c5pcj" event={"ID":"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4","Type":"ContainerStarted","Data":"4320531b8d8bb2fd61352c29c688400131c47bf1995b604feac331a8404f644d"}
Jul 13 21:12:36.508912 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:36.508875    2577 patch_prober.go:28] interesting pod/image-registry-567dfb87dc-cj76z container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 21:12:36.509074 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:36.508937    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" podUID="adac3e94-350e-48e3-a19b-fa521ffb5884" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:12:36.597619 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:36.597524    2577 generic.go:356] "Generic (PLEG): container finished" podID="ada5e595-ce6b-4486-8a21-60e7a7ffd4c4" containerID="22b9a4128edf3c4e90fe3444a0336688d9fdebef777d44e1b1a8bd4d6779e55e" exitCode=0
Jul 13 21:12:36.597619 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:36.597579    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c5pcj" event={"ID":"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4","Type":"ContainerDied","Data":"22b9a4128edf3c4e90fe3444a0336688d9fdebef777d44e1b1a8bd4d6779e55e"}
Jul 13 21:12:37.154403 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.154368    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:37.231126 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.231093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:37.291078 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.291034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:37.603966 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.603870    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c5pcj" event={"ID":"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4","Type":"ContainerStarted","Data":"ec88641cbc938e34e5e81dd66deb436866fc6335a1b90b5e3cd513f674aa1031"}
Jul 13 21:12:37.603966 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.603928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c5pcj" event={"ID":"ada5e595-ce6b-4486-8a21-60e7a7ffd4c4","Type":"ContainerStarted","Data":"a9d5f4b19d1581dce1ed6a798e0c4004c0d924aa1d8c0d5fa68f456cc55e5185"}
Jul 13 21:12:37.624564 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.624509    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c5pcj" podStartSLOduration=2.46972843 podStartE2EDuration="3.624492059s" totalImagesPullingTime="1.089402903s" totalInitContainerRuntime="65.360726ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:37.623969825 +0000 UTC m=+89.040896169" watchObservedRunningTime="2026-07-13 21:12:37.624492059 +0000 UTC m=+89.041418428"
Jul 13 21:12:37.665777 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:37.665717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:39.024512 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:39.024473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:39.088899 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:39.088863    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:39.650061 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:39.650020    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:40.495720 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:40.495688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:41.667623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:41.667591    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:41.729079 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:41.729046    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:43.464651 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:43.464614    2577 patch_prober.go:28] interesting pod/image-registry-567dfb87dc-cj76z container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 13 21:12:43.465143 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:43.464673    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z" podUID="adac3e94-350e-48e3-a19b-fa521ffb5884" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:12:43.601932 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:43.601894    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:43.702242 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:43.702210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:43.761442 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:43.761358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:45.111845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:45.111818    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:45.163110 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:45.163084    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:45.627530 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:45.627492    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:45.709538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:45.709507    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:46.508997 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:46.508966    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-567dfb87dc-cj76z"
Jul 13 21:12:46.621183 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:46.621150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:46.721707 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:46.721673    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:46.822546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:46.822459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:46.884257 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:46.884225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:47.803752 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:47.803715    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:47.866630 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:47.866602    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:49.339842 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:49.339808    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:49.438050 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:49.438019    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:50.826182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:50.826149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:50.927255 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:50.927226    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:51.531543 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:51.531511    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:51.631576 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:51.631532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:52.697912 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:52.697880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:52.798591 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:52.798557    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:53.545198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:53.545170    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-27v2r"
Jul 13 21:12:56.878878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:56.878842    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:12:56.938984 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:12:56.938952    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:04.943287 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:04.943244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:04.996576 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:04.996544    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:12.190383 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:12.190340    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8w765_edb8e8a1-8d39-4f54-9858-158909fbd6fb/serve-healthcheck-canary/0.log"
Jul 13 21:13:12.837464 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:12.837432    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:12.902595 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:12.902554    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:20.048332 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:20.048288    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:20.104529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:20.104499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:26.503052 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:26.503015    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:26.604682 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:26.604642    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:26.668539 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:26.668506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:53.464453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:53.464416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:13:53.933188 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:13:53.933151    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:02.934337 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:02.934258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:02.993437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:02.993402    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:03.081801 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:03.081754    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:03.679861 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:03.679830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:03.730375 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:03.730343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:08.716853 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:08.716814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:08.779872 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:08.779843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:12.982471 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:12.982435    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:13.082014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:13.081982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:13.837344 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:13.837306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:13.936208 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:13.936175    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:16.846493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:16.846464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:20.575342 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:20.575303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:20.636576 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:20.636541    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:24.202401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:24.202371    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:24.303947 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:24.303911    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:24.362330 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:24.362305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:34.977091 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:34.977050    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:35.072129 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:35.072095    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:42.601754 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:42.601716    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:14:42.702167 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:14:42.702139    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:01.998488 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:01.998455    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:02.081549 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:02.081513    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:25.867062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:25.867022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:32.799959 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:32.799873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:38.159204 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:38.159174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:43.605654 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:43.605614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:44.412112 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:44.412075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:47.253805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:47.253750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:47.354602 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:47.354572    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:48.604615 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:48.604576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:48.705531 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:48.705496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:49.483499 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:49.483467    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:49.591427 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:49.591382    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:51.699100 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:51.699072    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:52.084134 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:52.084055    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:54.010894 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:54.010859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:15:54.111193 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:15:54.111147    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:00.506836 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:00.506779    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:01.529980 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:01.529941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:01.629699 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:01.629647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:06.525048 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:06.524992    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:06.623208 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:06.623171    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:08.024686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:08.024643    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:08.129753 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:08.129712    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:09.060783 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:09.060741    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:16:09.064731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:09.064711    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 21:16:09.066227 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:09.066208    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:16:10.179890 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:10.179854    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:22.668331 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:22.668293    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:22.762794 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:22.762745    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:23.311164 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.311082    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:23.614111 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.613999    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:23.751699 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.751669    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:23.802282 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.802251    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"]
Jul 13 21:16:23.804469 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.804449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:23.809454 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.809427    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 21:16:23.809454 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.809445    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 21:16:23.809717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.809593    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 21:16:23.809717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.809705    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 21:16:23.809914 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.809899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q5mf8\""
Jul 13 21:16:23.834189 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.834149    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"]
Jul 13 21:16:23.957040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.957008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:23.966175 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.966147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:23.966260 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.966187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:23.966260 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.966212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:23.966339 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.966274    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:23.966339 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:23.966309    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cng7w\" (UniqueName: \"kubernetes.io/projected/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-kube-api-access-cng7w\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.042354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.042325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:24.067500 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.067618 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.067618 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.067618 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.067618 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cng7w\" (UniqueName: \"kubernetes.io/projected/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-kube-api-access-cng7w\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.068011 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.067989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.069880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.069860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.069982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.069949    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.070105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.070085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.075529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.075497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cng7w\" (UniqueName: \"kubernetes.io/projected/8eb07a66-8351-4e18-8dbe-3c3eb8c188b0-kube-api-access-cng7w\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-27m6v\" (UID: \"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.114420 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.114382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:24.142443 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.142408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:24.236331 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.236210    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:24.245581 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.245467    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"]
Jul 13 21:16:24.248526 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:16:24.248475    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb07a66_8351_4e18_8dbe_3c3eb8c188b0.slice/crio-888f4fa76ab622094c6e903670492fc00a9906e722e6be10422e763d4ebeb6df WatchSource:0}: Error finding container 888f4fa76ab622094c6e903670492fc00a9906e722e6be10422e763d4ebeb6df: Status 404 returned error can't find the container with id 888f4fa76ab622094c6e903670492fc00a9906e722e6be10422e763d4ebeb6df
Jul 13 21:16:24.250346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.250318    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:16:24.313943 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.313913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:24.331836 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.331804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v" event={"ID":"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0","Type":"ContainerStarted","Data":"888f4fa76ab622094c6e903670492fc00a9906e722e6be10422e763d4ebeb6df"}
Jul 13 21:16:24.412244 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:24.412212    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:28.349817 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:28.349780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v" event={"ID":"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0","Type":"ContainerStarted","Data":"c6d220de12f64549d9e01e20faf8017c50c1b8a392c40e0132be139f1a48dac7"}
Jul 13 21:16:29.354552 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:29.354516    2577 generic.go:356] "Generic (PLEG): container finished" podID="8eb07a66-8351-4e18-8dbe-3c3eb8c188b0" containerID="c6d220de12f64549d9e01e20faf8017c50c1b8a392c40e0132be139f1a48dac7" exitCode=0
Jul 13 21:16:29.354960 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:29.354608    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v" event={"ID":"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0","Type":"ContainerDied","Data":"c6d220de12f64549d9e01e20faf8017c50c1b8a392c40e0132be139f1a48dac7"}
Jul 13 21:16:30.360548 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:30.360504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v" event={"ID":"8eb07a66-8351-4e18-8dbe-3c3eb8c188b0","Type":"ContainerStarted","Data":"39ea72aed004088dcfd960816d925d7823167400373ce13c9fc5ce2d0f76081a"}
Jul 13 21:16:30.360940 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:30.360636    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:30.385825 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:30.385772    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v" podStartSLOduration=2.7207380370000003 podStartE2EDuration="7.385739115s" totalImagesPullingTime="3.659772437s" totalInitContainerRuntime="1.005228641s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:30.383731185 +0000 UTC m=+321.800657542" watchObservedRunningTime="2026-07-13 21:16:30.385739115 +0000 UTC m=+321.802665458"
Jul 13 21:16:41.366812 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:41.366754    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-27m6v"
Jul 13 21:16:43.699146 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:43.699100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:43.822847 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:43.822815    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:44.186022 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.185981    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:44.446573 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.446475    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:44.461332 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.461290    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"]
Jul 13 21:16:44.470965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.470932    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.475276 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.475252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 21:16:44.476215 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.476187    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 21:16:44.476215 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.476204    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tnsdf\""
Jul 13 21:16:44.476377 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.476283    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 21:16:44.476484 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.476468    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:16:44.479568 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.479540    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 21:16:44.489914 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.489886    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"]
Jul 13 21:16:44.527751 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.527713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81664d0b-8269-4fe5-b103-9e98259a8a29-manager-config\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.527937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.527779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.527937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.527806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-metrics-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.527937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.527872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwtpc\" (UniqueName: \"kubernetes.io/projected/81664d0b-8269-4fe5-b103-9e98259a8a29-kube-api-access-qwtpc\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.629125 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.629086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81664d0b-8269-4fe5-b103-9e98259a8a29-manager-config\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.629125 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.629130    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.629386 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.629154    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-metrics-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.629386 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.629187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwtpc\" (UniqueName: \"kubernetes.io/projected/81664d0b-8269-4fe5-b103-9e98259a8a29-kube-api-access-qwtpc\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.629950 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.629926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/81664d0b-8269-4fe5-b103-9e98259a8a29-manager-config\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.632203 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.632173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.632317 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.632238    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/81664d0b-8269-4fe5-b103-9e98259a8a29-metrics-cert\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.638389 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.638363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwtpc\" (UniqueName: \"kubernetes.io/projected/81664d0b-8269-4fe5-b103-9e98259a8a29-kube-api-access-qwtpc\") pod \"lws-controller-manager-575f94fb69-vbbdc\" (UID: \"81664d0b-8269-4fe5-b103-9e98259a8a29\") " pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.780744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.780662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:44.958790 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:44.958745    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"]
Jul 13 21:16:44.961440 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:16:44.961389    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81664d0b_8269_4fe5_b103_9e98259a8a29.slice/crio-e59078719cebc24d594b0eb278f4591f34bda124ea98b96228c78eaec4a970b0 WatchSource:0}: Error finding container e59078719cebc24d594b0eb278f4591f34bda124ea98b96228c78eaec4a970b0: Status 404 returned error can't find the container with id e59078719cebc24d594b0eb278f4591f34bda124ea98b96228c78eaec4a970b0
Jul 13 21:16:45.128369 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.128279    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:45.209842 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.209809    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:45.315120 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.315079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:45.355277 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.355241    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"]
Jul 13 21:16:45.358156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.358134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.360914 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.360889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 21:16:45.361066 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.361050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 21:16:45.361331 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.361317    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 21:16:45.361647 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.361622    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 21:16:45.361832 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.361814    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qlz4z\""
Jul 13 21:16:45.371852 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.371829    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"]
Jul 13 21:16:45.413285 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.413253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc" event={"ID":"81664d0b-8269-4fe5-b103-9e98259a8a29","Type":"ContainerStarted","Data":"e59078719cebc24d594b0eb278f4591f34bda124ea98b96228c78eaec4a970b0"}
Jul 13 21:16:45.436364 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.436324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5ef059-5257-4c47-a506-fea4867102bf-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.436571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.436374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5ef059-5257-4c47-a506-fea4867102bf-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.436571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.436393    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snq8z\" (UniqueName: \"kubernetes.io/projected/fb5ef059-5257-4c47-a506-fea4867102bf-kube-api-access-snq8z\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.537188 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.537148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5ef059-5257-4c47-a506-fea4867102bf-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.537391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.537209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5ef059-5257-4c47-a506-fea4867102bf-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.537391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.537241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snq8z\" (UniqueName: \"kubernetes.io/projected/fb5ef059-5257-4c47-a506-fea4867102bf-kube-api-access-snq8z\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.539557 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.539522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb5ef059-5257-4c47-a506-fea4867102bf-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.539722 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.539704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb5ef059-5257-4c47-a506-fea4867102bf-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.556848 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.556815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snq8z\" (UniqueName: \"kubernetes.io/projected/fb5ef059-5257-4c47-a506-fea4867102bf-kube-api-access-snq8z\") pod \"kube-auth-proxy-7cf89dc8f8-9ntvp\" (UID: \"fb5ef059-5257-4c47-a506-fea4867102bf\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.620293 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.620255    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:45.670960 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.670872    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"
Jul 13 21:16:45.747999 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.747962    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:45.828094 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.827935    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp"]
Jul 13 21:16:45.832527 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:16:45.832486    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5ef059_5257_4c47_a506_fea4867102bf.slice/crio-8f4909bbca50f9e559f2afadf7a5813bef4108e8a097318b9718d9505ce6123d WatchSource:0}: Error finding container 8f4909bbca50f9e559f2afadf7a5813bef4108e8a097318b9718d9505ce6123d: Status 404 returned error can't find the container with id 8f4909bbca50f9e559f2afadf7a5813bef4108e8a097318b9718d9505ce6123d
Jul 13 21:16:45.966745 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:45.966579    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:46.419099 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:46.419050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp" event={"ID":"fb5ef059-5257-4c47-a506-fea4867102bf","Type":"ContainerStarted","Data":"8f4909bbca50f9e559f2afadf7a5813bef4108e8a097318b9718d9505ce6123d"}
Jul 13 21:16:46.679412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:46.679333    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:46.801982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:46.801945    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:47.794537 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:47.794477    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:48.427774 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:48.427724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc" event={"ID":"81664d0b-8269-4fe5-b103-9e98259a8a29","Type":"ContainerStarted","Data":"d00a69dfc8051317e600d8023cad526aefd0a1182bf6e698f22bf1744a4611ba"}
Jul 13 21:16:48.427983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:48.427846    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:16:48.444522 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:48.444473    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc" podStartSLOduration=1.763406227 podStartE2EDuration="4.444457305s" totalImagesPullingTime="2.681051078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:48.443703235 +0000 UTC m=+339.860629580" watchObservedRunningTime="2026-07-13 21:16:48.444457305 +0000 UTC m=+339.861383649"
Jul 13 21:16:49.432506 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:49.432398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp" event={"ID":"fb5ef059-5257-4c47-a506-fea4867102bf","Type":"ContainerStarted","Data":"4286b0abcb8f2790c41d98f5055d71a912a3e3aeae98f6ae7cadb6c0ffb74af4"}
Jul 13 21:16:49.449300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:49.449243    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-9ntvp" podStartSLOduration=1.144489356 podStartE2EDuration="4.44922886s" totalImagesPullingTime="3.304739504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:49.448083075 +0000 UTC m=+340.865009423" watchObservedRunningTime="2026-07-13 21:16:49.44922886 +0000 UTC m=+340.866155203"
Jul 13 21:16:51.952632 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:51.952585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:52.054710 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:52.054667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:52.752500 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:52.752470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:53.010453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:53.010364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:53.262979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:53.262895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:55.111749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:55.111720    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:55.219915 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:55.219878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:55.293010 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:55.292977    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:55.520982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:55.520947    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:56.381919 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:56.381888    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:56.564258 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:56.564217    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:56.695709 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:56.695670    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:16:59.435627 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:16:59.435591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-575f94fb69-vbbdc"
Jul 13 21:17:14.206809 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:14.206707    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:16.154559 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:16.154531    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:45.061249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:45.061214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:45.146148 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:45.146112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:45.216279 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:45.216249    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:45.932742 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:45.932704    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:17:46.137631 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:17:46.137601    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:10.230744 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:10.230706    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:14.088037 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:14.087994    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:15.840199 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:15.840156    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:17.428033 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:17.428004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:17.829922 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:17.829830    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:19.279478 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:19.279443    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:20.843617 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:20.843580    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:21.681047 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:21.681007    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:21.787552 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:21.787522    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:23.665061 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:23.665029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:24.485671 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.485638    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"]
Jul 13 21:18:24.488706 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.488688    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.491293 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.491267    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 13 21:18:24.491472 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.491449    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:18:24.492178 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.492162    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:18:24.492274 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.492194    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 13 21:18:24.492274 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.492198    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v2h48\""
Jul 13 21:18:24.499291 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.499267    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"]
Jul 13 21:18:24.502362 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.502337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:24.593877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.593841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d561b14-d7da-4ad5-b242-9941805cbafc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.594058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.593882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch5qw\" (UniqueName: \"kubernetes.io/projected/7d561b14-d7da-4ad5-b242-9941805cbafc-kube-api-access-ch5qw\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.594058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.593948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d561b14-d7da-4ad5-b242-9941805cbafc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.694630 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.694595    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d561b14-d7da-4ad5-b242-9941805cbafc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.694630 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.694636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch5qw\" (UniqueName: \"kubernetes.io/projected/7d561b14-d7da-4ad5-b242-9941805cbafc-kube-api-access-ch5qw\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.695124 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.694664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d561b14-d7da-4ad5-b242-9941805cbafc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.695320 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.695298    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7d561b14-d7da-4ad5-b242-9941805cbafc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.697137 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.697109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d561b14-d7da-4ad5-b242-9941805cbafc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.705425 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.705390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch5qw\" (UniqueName: \"kubernetes.io/projected/7d561b14-d7da-4ad5-b242-9941805cbafc-kube-api-access-ch5qw\") pod \"kuadrant-console-plugin-66b5774985-dszdc\" (UID: \"7d561b14-d7da-4ad5-b242-9941805cbafc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.724601 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.724576    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:24.797886 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.797793    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"
Jul 13 21:18:24.921708 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:24.921686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc"]
Jul 13 21:18:24.924169 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:18:24.924141    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d561b14_d7da_4ad5_b242_9941805cbafc.slice/crio-1413c6524c7471e5ffdd9545d3e6188fc860844f5c9b2aa5bbee07b09076658d WatchSource:0}: Error finding container 1413c6524c7471e5ffdd9545d3e6188fc860844f5c9b2aa5bbee07b09076658d: Status 404 returned error can't find the container with id 1413c6524c7471e5ffdd9545d3e6188fc860844f5c9b2aa5bbee07b09076658d
Jul 13 21:18:25.442598 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:25.442564    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:25.543263 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:25.543234    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:25.770652 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:25.770560    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc" event={"ID":"7d561b14-d7da-4ad5-b242-9941805cbafc","Type":"ContainerStarted","Data":"1413c6524c7471e5ffdd9545d3e6188fc860844f5c9b2aa5bbee07b09076658d"}
Jul 13 21:18:33.050680 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:33.050594    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:33.154797 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:33.154749    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:37.935267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:37.935188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:38.031483 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:38.031444    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:38.141095 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:38.141066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:40.402157 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:40.402116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:40.490557 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:40.490525    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:45.848723 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:45.848676    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:45.949453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:45.949417    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:48.865467 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:48.865429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc" event={"ID":"7d561b14-d7da-4ad5-b242-9941805cbafc","Type":"ContainerStarted","Data":"720cf9c615b80c3f20f6889cd2dcf239a45736b56610091b5c32d13b2d966ddb"}
Jul 13 21:18:48.880970 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:48.880917    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dszdc" podStartSLOduration=1.5795969479999998 podStartE2EDuration="24.880900365s" totalImagesPullingTime="23.301303417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:48.880004727 +0000 UTC m=+460.296931071" watchObservedRunningTime="2026-07-13 21:18:48.880900365 +0000 UTC m=+460.297826709"
Jul 13 21:18:52.752448 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:52.752405    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:52.853320 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:52.853287    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:57.548620 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:57.548587    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:18:57.649612 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:18:57.649575    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:00.012265 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:00.012225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:00.137259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:00.137228    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:00.738364 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:00.738326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:03.762103 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:03.762066    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:04.174787 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:04.174732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:05.755002 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:05.754970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:05.808991 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:05.808965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:08.272065 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:08.272030    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:08.371977 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:08.371946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:08.472076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:08.472042    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:09.013453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:09.013422    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:09.064275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:09.064244    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:13.151297 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:13.151258    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:13.254857 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:13.254823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:17.759313 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:17.759275    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:17.859561 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:17.859526    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:18.466598 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:18.466563    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:18.572467 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:18.572437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:18.628533 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:18.628499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.254858 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.254826    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:19.284129 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.284062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.321590 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.321552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:19.321748 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.321661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:19.324099 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.324076    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rwl7d\""
Jul 13 21:19:19.365788 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.365745    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:19.366424 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.366401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.367956 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.367938    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:19.375796 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.375772    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:19.431360 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.431320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqds\" (UniqueName: \"kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds\") pod \"authorino-58cd955675-v6kvc\" (UID: \"9756cc0c-acfd-4199-b8b6-ba2043cd491d\") " pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:19.431531 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.431380    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5ccs\" (UniqueName: \"kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs\") pod \"authorino-75d556bd77-b6mc4\" (UID: \"a11e2f87-bd31-494d-b347-792385a64361\") " pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:19.532697 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.532603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqds\" (UniqueName: \"kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds\") pod \"authorino-58cd955675-v6kvc\" (UID: \"9756cc0c-acfd-4199-b8b6-ba2043cd491d\") " pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:19.532697 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.532660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5ccs\" (UniqueName: \"kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs\") pod \"authorino-75d556bd77-b6mc4\" (UID: \"a11e2f87-bd31-494d-b347-792385a64361\") " pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:19.548275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.548246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5ccs\" (UniqueName: \"kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs\") pod \"authorino-75d556bd77-b6mc4\" (UID: \"a11e2f87-bd31-494d-b347-792385a64361\") " pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:19.548404 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.548290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqds\" (UniqueName: \"kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds\") pod \"authorino-58cd955675-v6kvc\" (UID: \"9756cc0c-acfd-4199-b8b6-ba2043cd491d\") " pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:19.631155 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.631109    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:19.677286 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.677256    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:19.736983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.736895    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.785706 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.785564    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:19.788747 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:19:19.788717    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9756cc0c_acfd_4199_b8b6_ba2043cd491d.slice/crio-9bea5ec9e55f08c7b9f2b7cf316b16b4d0a0f621d7b5b722c8d5652b393269a7 WatchSource:0}: Error finding container 9bea5ec9e55f08c7b9f2b7cf316b16b4d0a0f621d7b5b722c8d5652b393269a7: Status 404 returned error can't find the container with id 9bea5ec9e55f08c7b9f2b7cf316b16b4d0a0f621d7b5b722c8d5652b393269a7
Jul 13 21:19:19.827346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.827311    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:19.830501 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:19:19.830472    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda11e2f87_bd31_494d_b347_792385a64361.slice/crio-044052fcab0674ee6362ea4c12a956e80ba4cecda3d1275efb2b28f0cd0226b5 WatchSource:0}: Error finding container 044052fcab0674ee6362ea4c12a956e80ba4cecda3d1275efb2b28f0cd0226b5: Status 404 returned error can't find the container with id 044052fcab0674ee6362ea4c12a956e80ba4cecda3d1275efb2b28f0cd0226b5
Jul 13 21:19:19.859756 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.859724    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.952173 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.952136    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:19.981233 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.981201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6mc4" event={"ID":"a11e2f87-bd31-494d-b347-792385a64361","Type":"ContainerStarted","Data":"044052fcab0674ee6362ea4c12a956e80ba4cecda3d1275efb2b28f0cd0226b5"}
Jul 13 21:19:19.982336 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:19.982306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v6kvc" event={"ID":"9756cc0c-acfd-4199-b8b6-ba2043cd491d","Type":"ContainerStarted","Data":"9bea5ec9e55f08c7b9f2b7cf316b16b4d0a0f621d7b5b722c8d5652b393269a7"}
Jul 13 21:19:20.219084 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:20.219043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:20.321958 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:20.321922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:24.001070 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:24.001027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6mc4" event={"ID":"a11e2f87-bd31-494d-b347-792385a64361","Type":"ContainerStarted","Data":"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"}
Jul 13 21:19:24.002489 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:24.002464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v6kvc" event={"ID":"9756cc0c-acfd-4199-b8b6-ba2043cd491d","Type":"ContainerStarted","Data":"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"}
Jul 13 21:19:24.015421 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:24.015371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-b6mc4" podStartSLOduration=1.5750492870000001 podStartE2EDuration="5.015356811s" totalImagesPullingTime="3.440307524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:24.014144944 +0000 UTC m=+495.431071289" watchObservedRunningTime="2026-07-13 21:19:24.015356811 +0000 UTC m=+495.432283155"
Jul 13 21:19:24.027218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:24.027162    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-v6kvc" podStartSLOduration=1.5279307549999999 podStartE2EDuration="5.027141204s" totalImagesPullingTime="3.499210449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:24.026643745 +0000 UTC m=+495.443570089" watchObservedRunningTime="2026-07-13 21:19:24.027141204 +0000 UTC m=+495.444067549"
Jul 13 21:19:24.051753 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:24.051724    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:26.010073 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:26.010031    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-v6kvc" podUID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" containerName="authorino" containerID="cri-o://65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136" gracePeriod=30
Jul 13 21:19:26.250118 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:26.250095    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:26.395306 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:26.395282    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsqds\" (UniqueName: \"kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds\") pod \"9756cc0c-acfd-4199-b8b6-ba2043cd491d\" (UID: \"9756cc0c-acfd-4199-b8b6-ba2043cd491d\") "
Jul 13 21:19:26.397395 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:26.397367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds" (OuterVolumeSpecName: "kube-api-access-xsqds") pod "9756cc0c-acfd-4199-b8b6-ba2043cd491d" (UID: "9756cc0c-acfd-4199-b8b6-ba2043cd491d"). InnerVolumeSpecName "kube-api-access-xsqds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:19:26.496722 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:26.496683    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsqds\" (UniqueName: \"kubernetes.io/projected/9756cc0c-acfd-4199-b8b6-ba2043cd491d-kube-api-access-xsqds\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:19:27.015059 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.015024    2577 generic.go:356] "Generic (PLEG): container finished" podID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" containerID="65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136" exitCode=0
Jul 13 21:19:27.015535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.015064    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v6kvc"
Jul 13 21:19:27.015535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.015108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v6kvc" event={"ID":"9756cc0c-acfd-4199-b8b6-ba2043cd491d","Type":"ContainerDied","Data":"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"}
Jul 13 21:19:27.015535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.015150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v6kvc" event={"ID":"9756cc0c-acfd-4199-b8b6-ba2043cd491d","Type":"ContainerDied","Data":"9bea5ec9e55f08c7b9f2b7cf316b16b4d0a0f621d7b5b722c8d5652b393269a7"}
Jul 13 21:19:27.015535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.015174    2577 scope.go:117] "RemoveContainer" containerID="65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"
Jul 13 21:19:27.026235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.026218    2577 scope.go:117] "RemoveContainer" containerID="65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"
Jul 13 21:19:27.026504 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:19:27.026487    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136\": container with ID starting with 65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136 not found: ID does not exist" containerID="65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"
Jul 13 21:19:27.026553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.026513    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136"} err="failed to get container status \"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136\": rpc error: code = NotFound desc = could not find container \"65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136\": container with ID starting with 65ec0cfbc2875f472c3f9b37572ecdf7faa01cd9b8ab87b3456472dd4503b136 not found: ID does not exist"
Jul 13 21:19:27.036509 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.036481    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:27.038545 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.038524    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-v6kvc"]
Jul 13 21:19:27.181891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:27.181861    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" path="/var/lib/kubelet/pods/9756cc0c-acfd-4199-b8b6-ba2043cd491d/volumes"
Jul 13 21:19:37.052207 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:37.052165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:37.151223 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:37.151188    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:37.535707 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:37.535672    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:49.797359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:49.797325    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:49.920916 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:49.920880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:50.021652 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.021621    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:50.021937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.021923    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" containerName="authorino"
Jul 13 21:19:50.021937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.021939    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" containerName="authorino"
Jul 13 21:19:50.022038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.022026    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9756cc0c-acfd-4199-b8b6-ba2043cd491d" containerName="authorino"
Jul 13 21:19:50.024107 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.024088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:50.030975 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.030951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:50.088837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.088728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6t7w\" (UniqueName: \"kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w\") pod \"authorino-5d4b67c446-kj848\" (UID: \"b29e8940-0ee9-4867-9fa9-811866f7c878\") " pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:50.189931 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.189899    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6t7w\" (UniqueName: \"kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w\") pod \"authorino-5d4b67c446-kj848\" (UID: \"b29e8940-0ee9-4867-9fa9-811866f7c878\") " pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:50.198551 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.198522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6t7w\" (UniqueName: \"kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w\") pod \"authorino-5d4b67c446-kj848\" (UID: \"b29e8940-0ee9-4867-9fa9-811866f7c878\") " pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:50.207457 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.207431    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:50.207659 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.207646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:50.233665 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.233625    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-b2sqf"]
Jul 13 21:19:50.236206 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.236180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.238337 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.238317    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 13 21:19:50.244402 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.244372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-b2sqf"]
Jul 13 21:19:50.282018 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.281990    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-b2sqf"]
Jul 13 21:19:50.282238 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:19:50.282215    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vrvwx tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-vrvwx tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-b2sqf" podUID="5f8ee65e-a77e-4a21-ab90-5f369a2d16fa"
Jul 13 21:19:50.290967 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.290935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.291124 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.290976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrvwx\" (UniqueName: \"kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.312359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.312326    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:19:50.314533 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.314510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.325703 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.325589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:19:50.339837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.339744    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:50.342790 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:19:50.342748    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb29e8940_0ee9_4867_9fa9_811866f7c878.slice/crio-843618e100703e13993827dd68a2fc5702f587b109d9b9930822cf0c8b1e9120 WatchSource:0}: Error finding container 843618e100703e13993827dd68a2fc5702f587b109d9b9930822cf0c8b1e9120: Status 404 returned error can't find the container with id 843618e100703e13993827dd68a2fc5702f587b109d9b9930822cf0c8b1e9120
Jul 13 21:19:50.392189 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.392149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4llrl\" (UniqueName: \"kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.392189 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.392191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.392399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.392219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.392399 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.392273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrvwx\" (UniqueName: \"kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.394686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.394657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.401054 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.401030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrvwx\" (UniqueName: \"kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx\") pod \"authorino-76877994-b2sqf\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") " pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:50.493651 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.493596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4llrl\" (UniqueName: \"kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.493651 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.493649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.496064 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.496036    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.501340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.501318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4llrl\" (UniqueName: \"kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl\") pod \"authorino-5479b47546-txl8g\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") " pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.623811 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.623705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:19:50.749471 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:50.749440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:19:50.752573 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:19:50.752548    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d4f4e3b_c0ab_4d84_9244_f1f3ecc197a0.slice/crio-bef1b9869f241c0c4512e69546f082e68e96e3a315874f93020fde7a1c3e6a36 WatchSource:0}: Error finding container bef1b9869f241c0c4512e69546f082e68e96e3a315874f93020fde7a1c3e6a36: Status 404 returned error can't find the container with id bef1b9869f241c0c4512e69546f082e68e96e3a315874f93020fde7a1c3e6a36
Jul 13 21:19:51.113831 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.113787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5479b47546-txl8g" event={"ID":"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0","Type":"ContainerStarted","Data":"bef1b9869f241c0c4512e69546f082e68e96e3a315874f93020fde7a1c3e6a36"}
Jul 13 21:19:51.116412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.116390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:51.116535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.116408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kj848" event={"ID":"b29e8940-0ee9-4867-9fa9-811866f7c878","Type":"ContainerStarted","Data":"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"}
Jul 13 21:19:51.116535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.116452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kj848" event={"ID":"b29e8940-0ee9-4867-9fa9-811866f7c878","Type":"ContainerStarted","Data":"843618e100703e13993827dd68a2fc5702f587b109d9b9930822cf0c8b1e9120"}
Jul 13 21:19:51.116535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.116478    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-kj848" podUID="b29e8940-0ee9-4867-9fa9-811866f7c878" containerName="authorino" containerID="cri-o://e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b" gracePeriod=30
Jul 13 21:19:51.122111 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.122091    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:51.130595 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.130554    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-kj848" podStartSLOduration=0.431022933 podStartE2EDuration="1.130539055s" totalImagesPullingTime="699.516122ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:51.129529762 +0000 UTC m=+522.546456117" watchObservedRunningTime="2026-07-13 21:19:51.130539055 +0000 UTC m=+522.547465398"
Jul 13 21:19:51.197264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.197244    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrvwx\" (UniqueName: \"kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx\") pod \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") "
Jul 13 21:19:51.197322 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.197291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert\") pod \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\" (UID: \"5f8ee65e-a77e-4a21-ab90-5f369a2d16fa\") "
Jul 13 21:19:51.199271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.199245    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx" (OuterVolumeSpecName: "kube-api-access-vrvwx") pod "5f8ee65e-a77e-4a21-ab90-5f369a2d16fa" (UID: "5f8ee65e-a77e-4a21-ab90-5f369a2d16fa"). InnerVolumeSpecName "kube-api-access-vrvwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:19:51.199271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.199251    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5f8ee65e-a77e-4a21-ab90-5f369a2d16fa" (UID: "5f8ee65e-a77e-4a21-ab90-5f369a2d16fa"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:19:51.298782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.298683    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-tls-cert\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:19:51.298782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.298708    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrvwx\" (UniqueName: \"kubernetes.io/projected/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa-kube-api-access-vrvwx\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:19:51.364947 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.364922    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:51.399826 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.399794    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6t7w\" (UniqueName: \"kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w\") pod \"b29e8940-0ee9-4867-9fa9-811866f7c878\" (UID: \"b29e8940-0ee9-4867-9fa9-811866f7c878\") "
Jul 13 21:19:51.401817 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.401786    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w" (OuterVolumeSpecName: "kube-api-access-f6t7w") pod "b29e8940-0ee9-4867-9fa9-811866f7c878" (UID: "b29e8940-0ee9-4867-9fa9-811866f7c878"). InnerVolumeSpecName "kube-api-access-f6t7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:19:51.500461 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:51.500427    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6t7w\" (UniqueName: \"kubernetes.io/projected/b29e8940-0ee9-4867-9fa9-811866f7c878-kube-api-access-f6t7w\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:19:52.122977 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.122938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5479b47546-txl8g" event={"ID":"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0","Type":"ContainerStarted","Data":"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"}
Jul 13 21:19:52.124199 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124172    2577 generic.go:356] "Generic (PLEG): container finished" podID="b29e8940-0ee9-4867-9fa9-811866f7c878" containerID="e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b" exitCode=2
Jul 13 21:19:52.124326 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124203    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-kj848"
Jul 13 21:19:52.124326 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kj848" event={"ID":"b29e8940-0ee9-4867-9fa9-811866f7c878","Type":"ContainerDied","Data":"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"}
Jul 13 21:19:52.124326 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124266    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-kj848" event={"ID":"b29e8940-0ee9-4867-9fa9-811866f7c878","Type":"ContainerDied","Data":"843618e100703e13993827dd68a2fc5702f587b109d9b9930822cf0c8b1e9120"}
Jul 13 21:19:52.124326 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124288    2577 scope.go:117] "RemoveContainer" containerID="e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"
Jul 13 21:19:52.124491 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.124419    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-b2sqf"
Jul 13 21:19:52.133414 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.133392    2577 scope.go:117] "RemoveContainer" containerID="e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"
Jul 13 21:19:52.133810 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:19:52.133788    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b\": container with ID starting with e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b not found: ID does not exist" containerID="e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"
Jul 13 21:19:52.133885 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.133819    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b"} err="failed to get container status \"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b\": rpc error: code = NotFound desc = could not find container \"e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b\": container with ID starting with e7e1f4cb47900fee77e9bbb99fb48c9a75e237f5d3f96535da40cc21b7d3640b not found: ID does not exist"
Jul 13 21:19:52.137732 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.137690    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5479b47546-txl8g" podStartSLOduration=1.711290663 podStartE2EDuration="2.137676186s" totalImagesPullingTime="426.385523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:52.136648082 +0000 UTC m=+523.553574428" watchObservedRunningTime="2026-07-13 21:19:52.137676186 +0000 UTC m=+523.554602529"
Jul 13 21:19:52.150651 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.150615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:52.154628 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.154599    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-kj848"]
Jul 13 21:19:52.160387 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.160356    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:52.160605 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.160570    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-b6mc4" podUID="a11e2f87-bd31-494d-b347-792385a64361" containerName="authorino" containerID="cri-o://b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d" gracePeriod=30
Jul 13 21:19:52.179171 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.179135    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-b2sqf"]
Jul 13 21:19:52.180627 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.180600    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-b2sqf"]
Jul 13 21:19:52.413691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.413669    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:52.508176 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.508135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5ccs\" (UniqueName: \"kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs\") pod \"a11e2f87-bd31-494d-b347-792385a64361\" (UID: \"a11e2f87-bd31-494d-b347-792385a64361\") "
Jul 13 21:19:52.510232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.510204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs" (OuterVolumeSpecName: "kube-api-access-m5ccs") pod "a11e2f87-bd31-494d-b347-792385a64361" (UID: "a11e2f87-bd31-494d-b347-792385a64361"). InnerVolumeSpecName "kube-api-access-m5ccs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:19:52.608859 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:52.608821    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5ccs\" (UniqueName: \"kubernetes.io/projected/a11e2f87-bd31-494d-b347-792385a64361-kube-api-access-m5ccs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:19:53.128989 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.128950    2577 generic.go:356] "Generic (PLEG): container finished" podID="a11e2f87-bd31-494d-b347-792385a64361" containerID="b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d" exitCode=0
Jul 13 21:19:53.129406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.128995    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6mc4"
Jul 13 21:19:53.129406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.129029    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6mc4" event={"ID":"a11e2f87-bd31-494d-b347-792385a64361","Type":"ContainerDied","Data":"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"}
Jul 13 21:19:53.129406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.129079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6mc4" event={"ID":"a11e2f87-bd31-494d-b347-792385a64361","Type":"ContainerDied","Data":"044052fcab0674ee6362ea4c12a956e80ba4cecda3d1275efb2b28f0cd0226b5"}
Jul 13 21:19:53.129406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.129105    2577 scope.go:117] "RemoveContainer" containerID="b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"
Jul 13 21:19:53.137311 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.137291    2577 scope.go:117] "RemoveContainer" containerID="b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"
Jul 13 21:19:53.137597 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:19:53.137579    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d\": container with ID starting with b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d not found: ID does not exist" containerID="b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"
Jul 13 21:19:53.137656 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.137609    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d"} err="failed to get container status \"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d\": rpc error: code = NotFound desc = could not find container \"b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d\": container with ID starting with b26edd7ddd0235abfbe7d16f08e4972bf96280c143d1d892f7935772ae13d67d not found: ID does not exist"
Jul 13 21:19:53.152335 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.152301    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:53.156017 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.155992    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6mc4"]
Jul 13 21:19:53.181845 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.181814    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f8ee65e-a77e-4a21-ab90-5f369a2d16fa" path="/var/lib/kubelet/pods/5f8ee65e-a77e-4a21-ab90-5f369a2d16fa/volumes"
Jul 13 21:19:53.182043 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.182030    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a11e2f87-bd31-494d-b347-792385a64361" path="/var/lib/kubelet/pods/a11e2f87-bd31-494d-b347-792385a64361/volumes"
Jul 13 21:19:53.182308 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:53.182296    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b29e8940-0ee9-4867-9fa9-811866f7c878" path="/var/lib/kubelet/pods/b29e8940-0ee9-4867-9fa9-811866f7c878/volumes"
Jul 13 21:19:57.000206 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.000174    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:57.092694 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.092653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:57.153719 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.153693    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:57.182518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182489    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:19:57.182816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182776    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a11e2f87-bd31-494d-b347-792385a64361" containerName="authorino"
Jul 13 21:19:57.182816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182790    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a11e2f87-bd31-494d-b347-792385a64361" containerName="authorino"
Jul 13 21:19:57.182816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182798    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b29e8940-0ee9-4867-9fa9-811866f7c878" containerName="authorino"
Jul 13 21:19:57.182816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182803    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b29e8940-0ee9-4867-9fa9-811866f7c878" containerName="authorino"
Jul 13 21:19:57.183068 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182851    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a11e2f87-bd31-494d-b347-792385a64361" containerName="authorino"
Jul 13 21:19:57.183068 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.182859    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b29e8940-0ee9-4867-9fa9-811866f7c878" containerName="authorino"
Jul 13 21:19:57.187476 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.187455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.191309 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.191284    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 13 21:19:57.191309 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.191300    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 13 21:19:57.191453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.191333    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vmxct\""
Jul 13 21:19:57.195693 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.195659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:19:57.229287 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.229256    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:57.243344 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.243316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw848\" (UniqueName: \"kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.243432 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.243372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.344644 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.344560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.344644 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.344618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw848\" (UniqueName: \"kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.346811 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.346756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.352625 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.352599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw848\" (UniqueName: \"kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848\") pod \"maas-controller-5988d4cf74-trb5q\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") " pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.497704 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.497662    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:19:57.644939 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:57.644912    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:19:57.647181 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:19:57.647153    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839b0717_f79b_4130_9034_a818b8feab22.slice/crio-1a02b5620aa6ca86a58787e33ce07d7e7c01deac2a1d116496ae796d296c4d16 WatchSource:0}: Error finding container 1a02b5620aa6ca86a58787e33ce07d7e7c01deac2a1d116496ae796d296c4d16: Status 404 returned error can't find the container with id 1a02b5620aa6ca86a58787e33ce07d7e7c01deac2a1d116496ae796d296c4d16
Jul 13 21:19:58.152944 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:58.152911    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-trb5q" event={"ID":"839b0717-f79b-4130-9034-a818b8feab22","Type":"ContainerStarted","Data":"1a02b5620aa6ca86a58787e33ce07d7e7c01deac2a1d116496ae796d296c4d16"}
Jul 13 21:19:59.811218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:59.811176    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:19:59.908346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:19:59.908310    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:00.163667 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:00.163623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-trb5q" event={"ID":"839b0717-f79b-4130-9034-a818b8feab22","Type":"ContainerStarted","Data":"221ab1403b60376acd849feeb20ae985fb8d1805a636fdbdc0db572288a05535"}
Jul 13 21:20:00.163853 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:00.163755    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:20:00.181235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:00.180996    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5988d4cf74-trb5q" podStartSLOduration=0.752082455 podStartE2EDuration="3.180974759s" totalImagesPullingTime="2.428892304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:20:00.179825176 +0000 UTC m=+531.596751521" watchObservedRunningTime="2026-07-13 21:20:00.180974759 +0000 UTC m=+531.597901105"
Jul 13 21:20:11.171185 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:11.171148    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:20:31.881957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:31.881908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:31.977016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:31.976982    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:32.418104 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:32.418063    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:32.756066 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:32.755978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:32.860054 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:32.860022    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:32.935392 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:32.935362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:33.042585 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.042496    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"]
Jul 13 21:20:33.046646 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.046620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.048965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.048940    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 13 21:20:33.049082 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.048943    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hq54l\""
Jul 13 21:20:33.055580 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.055553    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"]
Jul 13 21:20:33.147859 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.147810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5scb\" (UniqueName: \"kubernetes.io/projected/337a357f-6414-4e0e-be7b-b32abebd8b0c-kube-api-access-c5scb\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.148067 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.147881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a357f-6414-4e0e-be7b-b32abebd8b0c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.248480 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.248439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5scb\" (UniqueName: \"kubernetes.io/projected/337a357f-6414-4e0e-be7b-b32abebd8b0c-kube-api-access-c5scb\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.248657 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.248493    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a357f-6414-4e0e-be7b-b32abebd8b0c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.249365 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.249340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/337a357f-6414-4e0e-be7b-b32abebd8b0c-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.256145 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.256118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5scb\" (UniqueName: \"kubernetes.io/projected/337a357f-6414-4e0e-be7b-b32abebd8b0c-kube-api-access-c5scb\") pod \"payload-pre-processing-74584bb9f8-qzqz6\" (UID: \"337a357f-6414-4e0e-be7b-b32abebd8b0c\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.357680 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.357589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:33.487856 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:33.487820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"]
Jul 13 21:20:33.491042 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:20:33.491000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337a357f_6414_4e0e_be7b_b32abebd8b0c.slice/crio-b39c0cf4ffd56c42c64a005989f57ce00d28423186310324f72f052f0f5df7c7 WatchSource:0}: Error finding container b39c0cf4ffd56c42c64a005989f57ce00d28423186310324f72f052f0f5df7c7: Status 404 returned error can't find the container with id b39c0cf4ffd56c42c64a005989f57ce00d28423186310324f72f052f0f5df7c7
Jul 13 21:20:34.302280 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:34.302243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6" event={"ID":"337a357f-6414-4e0e-be7b-b32abebd8b0c","Type":"ContainerStarted","Data":"b39c0cf4ffd56c42c64a005989f57ce00d28423186310324f72f052f0f5df7c7"}
Jul 13 21:20:35.940835 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:35.940800    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:36.312909 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:36.312821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6" event={"ID":"337a357f-6414-4e0e-be7b-b32abebd8b0c","Type":"ContainerStarted","Data":"20e36443303ccc7418479d6dcc37142e90a2b3ab0455afdf8ced4cf3dee97600"}
Jul 13 21:20:36.313101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:36.312954    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:20:36.329232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:36.329183    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6" podStartSLOduration=1.328309806 podStartE2EDuration="3.329164511s" totalImagesPullingTime="2.000854705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:20:36.328237204 +0000 UTC m=+567.745163549" watchObservedRunningTime="2026-07-13 21:20:36.329164511 +0000 UTC m=+567.746090855"
Jul 13 21:20:42.896717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:42.896684    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:43.007854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:43.007814    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:20:47.318507 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:20:47.318474    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-qzqz6"
Jul 13 21:21:00.558590 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.558549    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"]
Jul 13 21:21:00.563837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.563804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.568017 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.567959    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 21:21:00.568172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.567963    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sm8dp\""
Jul 13 21:21:00.568172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.567963    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 13 21:21:00.568172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.568122    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 21:21:00.571016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.570995    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"]
Jul 13 21:21:00.587258 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.587221    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:00.655142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655109    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8195d77e-7de4-46c2-be92-a7be07977460-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.655142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.655356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.655356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.655356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.655356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.655249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djlw7\" (UniqueName: \"kubernetes.io/projected/8195d77e-7de4-46c2-be92-a7be07977460-kube-api-access-djlw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756562 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756528    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8195d77e-7de4-46c2-be92-a7be07977460-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756562 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756843 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756843 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756843 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.756843 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.756706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djlw7\" (UniqueName: \"kubernetes.io/projected/8195d77e-7de4-46c2-be92-a7be07977460-kube-api-access-djlw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.757143 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.757117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.757229 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.757155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.757229 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.757167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.758924 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.758899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8195d77e-7de4-46c2-be92-a7be07977460-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.759106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.759087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8195d77e-7de4-46c2-be92-a7be07977460-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.765086 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.765061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djlw7\" (UniqueName: \"kubernetes.io/projected/8195d77e-7de4-46c2-be92-a7be07977460-kube-api-access-djlw7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm\" (UID: \"8195d77e-7de4-46c2-be92-a7be07977460\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:00.876907 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:00.876800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:01.008557 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:01.008517    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"]
Jul 13 21:21:01.011784 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:21:01.011746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8195d77e_7de4_46c2_be92_a7be07977460.slice/crio-d9a2a1087b55b367b5d2fd82b873acb5a33f4ea357afbf9a966f3fca84d52369 WatchSource:0}: Error finding container d9a2a1087b55b367b5d2fd82b873acb5a33f4ea357afbf9a966f3fca84d52369: Status 404 returned error can't find the container with id d9a2a1087b55b367b5d2fd82b873acb5a33f4ea357afbf9a966f3fca84d52369
Jul 13 21:21:01.414644 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:01.414582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm" event={"ID":"8195d77e-7de4-46c2-be92-a7be07977460","Type":"ContainerStarted","Data":"d9a2a1087b55b367b5d2fd82b873acb5a33f4ea357afbf9a966f3fca84d52369"}
Jul 13 21:21:02.419981 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:02.419943    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm" event={"ID":"8195d77e-7de4-46c2-be92-a7be07977460","Type":"ContainerStarted","Data":"482ebb93e8ee5f9c5501b28c88c81d7aaa62a19534097fa6005b21786173473b"}
Jul 13 21:21:02.437006 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:02.436948    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm" podStartSLOduration=1.326470142 podStartE2EDuration="2.436931574s" totalImagesPullingTime="1.110461432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:21:02.435685143 +0000 UTC m=+593.852611487" watchObservedRunningTime="2026-07-13 21:21:02.436931574 +0000 UTC m=+593.853857918"
Jul 13 21:21:07.361982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:07.361951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:09.087165 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:09.087138    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:21:09.089431 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:09.089410    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:21:09.780062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:09.780023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:10.877662 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:10.877617    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:10.880222 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:10.877726    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:10.890341 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:10.890317    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:11.469592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:11.469563    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm"
Jul 13 21:21:24.506722 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:24.506685    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:29.680364 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:29.680332    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:36.660926 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:36.660889    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:36.761924 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:36.761883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:38.045386 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:38.045350    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:38.096529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:38.096500    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:45.893454 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:45.893416    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:49.852643 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.852609    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"]
Jul 13 21:21:49.857590 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.857564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.860058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.860035    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 21:21:49.864091 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtf8t\" (UniqueName: \"kubernetes.io/projected/b763bfbf-63d7-41b6-8222-5e0ae402ee82-kube-api-access-rtf8t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864237 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864131    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864237 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864354 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864318    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.864735 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.864710    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"]
Jul 13 21:21:49.875481 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.875447    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:49.965595 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965556    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.965595 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.965880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965622    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.965880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtf8t\" (UniqueName: \"kubernetes.io/projected/b763bfbf-63d7-41b6-8222-5e0ae402ee82-kube-api-access-rtf8t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.965880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965720    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.965880 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.965787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.966115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.966091    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.966179 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.966135    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.966179 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.966155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.968089 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.968068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b763bfbf-63d7-41b6-8222-5e0ae402ee82-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.968262 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.968245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b763bfbf-63d7-41b6-8222-5e0ae402ee82-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:49.974150 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:49.974124    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtf8t\" (UniqueName: \"kubernetes.io/projected/b763bfbf-63d7-41b6-8222-5e0ae402ee82-kube-api-access-rtf8t\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5k9zp\" (UID: \"b763bfbf-63d7-41b6-8222-5e0ae402ee82\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:50.169200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.169161    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:21:50.300581 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.300558    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"]
Jul 13 21:21:50.303086 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:21:50.303060    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb763bfbf_63d7_41b6_8222_5e0ae402ee82.slice/crio-a50c37eb8335f227c05ee7699c5be7274738aa630c1af704a3a9c37beee16abe WatchSource:0}: Error finding container a50c37eb8335f227c05ee7699c5be7274738aa630c1af704a3a9c37beee16abe: Status 404 returned error can't find the container with id a50c37eb8335f227c05ee7699c5be7274738aa630c1af704a3a9c37beee16abe
Jul 13 21:21:50.304478 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.304454    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:21:50.621293 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.621237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp" event={"ID":"b763bfbf-63d7-41b6-8222-5e0ae402ee82","Type":"ContainerStarted","Data":"45d704fa0c996c4dc5561ed5c25ab4616049e478a7ce60d0001b4422ed1e8e11"}
Jul 13 21:21:50.621455 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.621352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp" event={"ID":"b763bfbf-63d7-41b6-8222-5e0ae402ee82","Type":"ContainerStarted","Data":"a50c37eb8335f227c05ee7699c5be7274738aa630c1af704a3a9c37beee16abe"}
Jul 13 21:21:50.637954 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:50.637894    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp" podStartSLOduration=1.423393132 podStartE2EDuration="1.63787651s" totalImagesPullingTime="214.483378ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:21:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:21:50.636906238 +0000 UTC m=+642.053832604" watchObservedRunningTime="2026-07-13 21:21:50.63787651 +0000 UTC m=+642.054802854"
Jul 13 21:21:59.843919 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.843882    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"]
Jul 13 21:21:59.847376 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.847352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.850119 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.850092    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 13 21:21:59.856488 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.856460    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"]
Jul 13 21:21:59.874728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.874695    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:21:59.953412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0578bf0-a990-480a-ad6c-2baee5564d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.953412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.953609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddzmt\" (UniqueName: \"kubernetes.io/projected/c0578bf0-a990-480a-ad6c-2baee5564d29-kube-api-access-ddzmt\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.953609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.953609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:21:59.953609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:21:59.953535    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054701 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054657    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054701 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzmt\" (UniqueName: \"kubernetes.io/projected/c0578bf0-a990-480a-ad6c-2baee5564d29-kube-api-access-ddzmt\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054779    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.054942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.054856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0578bf0-a990-480a-ad6c-2baee5564d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.055184 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.055160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.055239 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.055198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.055275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.055245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.056995 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.056975    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c0578bf0-a990-480a-ad6c-2baee5564d29-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.057251 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.057234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c0578bf0-a990-480a-ad6c-2baee5564d29-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.063267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.063234    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzmt\" (UniqueName: \"kubernetes.io/projected/c0578bf0-a990-480a-ad6c-2baee5564d29-kube-api-access-ddzmt\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl\" (UID: \"c0578bf0-a990-480a-ad6c-2baee5564d29\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.157384 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.157352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:00.169477 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.169450    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:22:00.169584 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.169512    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:22:00.186120 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.186092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:22:00.302088 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.302051    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"]
Jul 13 21:22:00.305722 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:22:00.305691    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc0578bf0_a990_480a_ad6c_2baee5564d29.slice/crio-4f382594476e3a10e0259cbd163c23cb5f45d2cc994933eeb94f38ab1e55382f WatchSource:0}: Error finding container 4f382594476e3a10e0259cbd163c23cb5f45d2cc994933eeb94f38ab1e55382f: Status 404 returned error can't find the container with id 4f382594476e3a10e0259cbd163c23cb5f45d2cc994933eeb94f38ab1e55382f
Jul 13 21:22:00.663041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.663004    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl" event={"ID":"c0578bf0-a990-480a-ad6c-2baee5564d29","Type":"ContainerStarted","Data":"d2f68c1679c75a982f21da1a3cfd46e8f078a2f0b0dd0cb3eba3d575b373f5cb"}
Jul 13 21:22:00.663041 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.663056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl" event={"ID":"c0578bf0-a990-480a-ad6c-2baee5564d29","Type":"ContainerStarted","Data":"4f382594476e3a10e0259cbd163c23cb5f45d2cc994933eeb94f38ab1e55382f"}
Jul 13 21:22:00.674611 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.674586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5k9zp"
Jul 13 21:22:00.679778 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:00.679721    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl" podStartSLOduration=1.506899852 podStartE2EDuration="1.679706457s" totalImagesPullingTime="172.806605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:22:00.6781296 +0000 UTC m=+652.095055944" watchObservedRunningTime="2026-07-13 21:22:00.679706457 +0000 UTC m=+652.096632801"
Jul 13 21:22:10.157854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:10.157813    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:10.157854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:10.157860    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:10.170342 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:10.170313    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:10.716139 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:10.716107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl"
Jul 13 21:22:16.097316 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:22:16.097280    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:22.183889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.183798    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b5dd9dbdc-7csqh"]
Jul 13 21:23:22.187257 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.187233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.194643 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.194618    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b5dd9dbdc-7csqh"]
Jul 13 21:23:22.275957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.275913    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/31f13755-f435-459a-b1b6-c4680209ed7f-tls-cert\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.276142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.275982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lqs5\" (UniqueName: \"kubernetes.io/projected/31f13755-f435-459a-b1b6-c4680209ed7f-kube-api-access-9lqs5\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.376866 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.376827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lqs5\" (UniqueName: \"kubernetes.io/projected/31f13755-f435-459a-b1b6-c4680209ed7f-kube-api-access-9lqs5\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.377046 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.376932    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/31f13755-f435-459a-b1b6-c4680209ed7f-tls-cert\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.379419 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.379390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/31f13755-f435-459a-b1b6-c4680209ed7f-tls-cert\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.384451 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.384422    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lqs5\" (UniqueName: \"kubernetes.io/projected/31f13755-f435-459a-b1b6-c4680209ed7f-kube-api-access-9lqs5\") pod \"authorino-7b5dd9dbdc-7csqh\" (UID: \"31f13755-f435-459a-b1b6-c4680209ed7f\") " pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.496718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.496608    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh"
Jul 13 21:23:22.625253 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.625203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b5dd9dbdc-7csqh"]
Jul 13 21:23:22.628945 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:23:22.628914    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31f13755_f435_459a_b1b6_c4680209ed7f.slice/crio-8f1eca49582bb7f18809e8cd859edb630e44646dfb3a87fa6b654a616404ed42 WatchSource:0}: Error finding container 8f1eca49582bb7f18809e8cd859edb630e44646dfb3a87fa6b654a616404ed42: Status 404 returned error can't find the container with id 8f1eca49582bb7f18809e8cd859edb630e44646dfb3a87fa6b654a616404ed42
Jul 13 21:23:22.999392 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:22.999354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh" event={"ID":"31f13755-f435-459a-b1b6-c4680209ed7f","Type":"ContainerStarted","Data":"8f1eca49582bb7f18809e8cd859edb630e44646dfb3a87fa6b654a616404ed42"}
Jul 13 21:23:24.006648 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.006617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh" event={"ID":"31f13755-f435-459a-b1b6-c4680209ed7f","Type":"ContainerStarted","Data":"85f7c40e11f810530037bcf758cd99d4764d1a00785f45c00873a4feaf27cc3f"}
Jul 13 21:23:24.022974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.022922    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b5dd9dbdc-7csqh" podStartSLOduration=1.627802828 podStartE2EDuration="2.02290337s" totalImagesPullingTime="395.100542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:23:24.021721474 +0000 UTC m=+735.438647817" watchObservedRunningTime="2026-07-13 21:23:24.02290337 +0000 UTC m=+735.439829713"
Jul 13 21:23:24.048718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.048680    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:23:24.048979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.048918    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5479b47546-txl8g" podUID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" containerName="authorino" containerID="cri-o://c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e" gracePeriod=30
Jul 13 21:23:24.304566 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.304537    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:23:24.389968 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.389929    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert\") pod \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") "
Jul 13 21:23:24.390158 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.390004    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4llrl\" (UniqueName: \"kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl\") pod \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\" (UID: \"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0\") "
Jul 13 21:23:24.392053 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.392024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl" (OuterVolumeSpecName: "kube-api-access-4llrl") pod "1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" (UID: "1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0"). InnerVolumeSpecName "kube-api-access-4llrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:23:24.400381 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.400353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" (UID: "1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:23:24.490942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.490903    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4llrl\" (UniqueName: \"kubernetes.io/projected/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-kube-api-access-4llrl\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:23:24.490942 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.490936    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0-tls-cert\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:23:24.630072 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.629984    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:24.729910 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:24.729874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:25.012331 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.012297    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" containerID="c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e" exitCode=0
Jul 13 21:23:25.012741 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.012350    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5479b47546-txl8g"
Jul 13 21:23:25.012741 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.012394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5479b47546-txl8g" event={"ID":"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0","Type":"ContainerDied","Data":"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"}
Jul 13 21:23:25.012741 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.012443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5479b47546-txl8g" event={"ID":"1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0","Type":"ContainerDied","Data":"bef1b9869f241c0c4512e69546f082e68e96e3a315874f93020fde7a1c3e6a36"}
Jul 13 21:23:25.012741 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.012473    2577 scope.go:117] "RemoveContainer" containerID="c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"
Jul 13 21:23:25.021725 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.021706    2577 scope.go:117] "RemoveContainer" containerID="c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"
Jul 13 21:23:25.022007 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:23:25.021989    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e\": container with ID starting with c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e not found: ID does not exist" containerID="c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"
Jul 13 21:23:25.022063 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.022015    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e"} err="failed to get container status \"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e\": rpc error: code = NotFound desc = could not find container \"c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e\": container with ID starting with c6c0f9d8a8cd1617deca5c3ec21e3fce99e27fd6e9392e8fcdc0dba3a5d6224e not found: ID does not exist"
Jul 13 21:23:25.033694 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.033667    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:23:25.045284 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.045258    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5479b47546-txl8g"]
Jul 13 21:23:25.187231 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.183442    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" path="/var/lib/kubelet/pods/1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0/volumes"
Jul 13 21:23:25.830665 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.830630    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:25.928576 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:25.928548    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:32.816405 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:32.816362    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:23:44.548350 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:23:44.548305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:24:38.024390 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.024307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:24:38.132932 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.132900    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:24:38.133156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.133129    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5988d4cf74-trb5q" podUID="839b0717-f79b-4130-9034-a818b8feab22" containerName="manager" containerID="cri-o://221ab1403b60376acd849feeb20ae985fb8d1805a636fdbdc0db572288a05535" gracePeriod=30
Jul 13 21:24:38.317631 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.317601    2577 generic.go:356] "Generic (PLEG): container finished" podID="839b0717-f79b-4130-9034-a818b8feab22" containerID="221ab1403b60376acd849feeb20ae985fb8d1805a636fdbdc0db572288a05535" exitCode=0
Jul 13 21:24:38.317816 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.317640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-trb5q" event={"ID":"839b0717-f79b-4130-9034-a818b8feab22","Type":"ContainerDied","Data":"221ab1403b60376acd849feeb20ae985fb8d1805a636fdbdc0db572288a05535"}
Jul 13 21:24:38.377891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.377860    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:24:38.406409 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.406378    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw848\" (UniqueName: \"kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848\") pod \"839b0717-f79b-4130-9034-a818b8feab22\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") "
Jul 13 21:24:38.406592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.406441    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert\") pod \"839b0717-f79b-4130-9034-a818b8feab22\" (UID: \"839b0717-f79b-4130-9034-a818b8feab22\") "
Jul 13 21:24:38.408611 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.408574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848" (OuterVolumeSpecName: "kube-api-access-jw848") pod "839b0717-f79b-4130-9034-a818b8feab22" (UID: "839b0717-f79b-4130-9034-a818b8feab22"). InnerVolumeSpecName "kube-api-access-jw848". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:24:38.408611 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.408603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "839b0717-f79b-4130-9034-a818b8feab22" (UID: "839b0717-f79b-4130-9034-a818b8feab22"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:24:38.508016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.507966    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw848\" (UniqueName: \"kubernetes.io/projected/839b0717-f79b-4130-9034-a818b8feab22-kube-api-access-jw848\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:24:38.508016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:38.508010    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/839b0717-f79b-4130-9034-a818b8feab22-webhook-cert\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:24:39.322697 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:39.322665    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-trb5q"
Jul 13 21:24:39.322697 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:39.322676    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-trb5q" event={"ID":"839b0717-f79b-4130-9034-a818b8feab22","Type":"ContainerDied","Data":"1a02b5620aa6ca86a58787e33ce07d7e7c01deac2a1d116496ae796d296c4d16"}
Jul 13 21:24:39.323213 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:39.322730    2577 scope.go:117] "RemoveContainer" containerID="221ab1403b60376acd849feeb20ae985fb8d1805a636fdbdc0db572288a05535"
Jul 13 21:24:39.338318 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:39.338296    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:24:39.343869 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:39.343844    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-trb5q"]
Jul 13 21:24:40.019133 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.019100    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:24:40.139466 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5988d4cf74-28fvc"]
Jul 13 21:24:40.139782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139745    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="839b0717-f79b-4130-9034-a818b8feab22" containerName="manager"
Jul 13 21:24:40.139782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139774    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="839b0717-f79b-4130-9034-a818b8feab22" containerName="manager"
Jul 13 21:24:40.139878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139795    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" containerName="authorino"
Jul 13 21:24:40.139878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139801    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" containerName="authorino"
Jul 13 21:24:40.139878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139844    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d4f4e3b-c0ab-4d84-9244-f1f3ecc197a0" containerName="authorino"
Jul 13 21:24:40.139878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.139855    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="839b0717-f79b-4130-9034-a818b8feab22" containerName="manager"
Jul 13 21:24:40.142473 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.142455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.144837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.144811    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 13 21:24:40.144983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.144843    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 13 21:24:40.145729 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.145716    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vmxct\""
Jul 13 21:24:40.150561 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.150537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-28fvc"]
Jul 13 21:24:40.223729 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.223695    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7a28d92-74ca-4712-a0ad-52307374782c-webhook-cert\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.223925 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.223862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rp2f\" (UniqueName: \"kubernetes.io/projected/c7a28d92-74ca-4712-a0ad-52307374782c-kube-api-access-9rp2f\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.324362 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.324272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7a28d92-74ca-4712-a0ad-52307374782c-webhook-cert\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.324808 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.324365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rp2f\" (UniqueName: \"kubernetes.io/projected/c7a28d92-74ca-4712-a0ad-52307374782c-kube-api-access-9rp2f\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.326711 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.326685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c7a28d92-74ca-4712-a0ad-52307374782c-webhook-cert\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.332076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.332053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rp2f\" (UniqueName: \"kubernetes.io/projected/c7a28d92-74ca-4712-a0ad-52307374782c-kube-api-access-9rp2f\") pod \"maas-controller-5988d4cf74-28fvc\" (UID: \"c7a28d92-74ca-4712-a0ad-52307374782c\") " pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.453198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.453158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:40.587038 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:40.586834    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5988d4cf74-28fvc"]
Jul 13 21:24:40.589778 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:24:40.589733    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7a28d92_74ca_4712_a0ad_52307374782c.slice/crio-9d7aa27a397874bb0e30f8a6170fb37dc16d8df3f8cf0d6abb79c0281ef39ce5 WatchSource:0}: Error finding container 9d7aa27a397874bb0e30f8a6170fb37dc16d8df3f8cf0d6abb79c0281ef39ce5: Status 404 returned error can't find the container with id 9d7aa27a397874bb0e30f8a6170fb37dc16d8df3f8cf0d6abb79c0281ef39ce5
Jul 13 21:24:41.183687 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:41.183643    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="839b0717-f79b-4130-9034-a818b8feab22" path="/var/lib/kubelet/pods/839b0717-f79b-4130-9034-a818b8feab22/volumes"
Jul 13 21:24:41.336743 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:41.336707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-28fvc" event={"ID":"c7a28d92-74ca-4712-a0ad-52307374782c","Type":"ContainerStarted","Data":"9d7aa27a397874bb0e30f8a6170fb37dc16d8df3f8cf0d6abb79c0281ef39ce5"}
Jul 13 21:24:42.343097 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:42.343058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5988d4cf74-28fvc" event={"ID":"c7a28d92-74ca-4712-a0ad-52307374782c","Type":"ContainerStarted","Data":"b73882858f55cb50f0d4fa402b2d5977e9909c788a6d156ba4756a3fa720b9f1"}
Jul 13 21:24:42.343575 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:42.343132    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:24:42.359645 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:42.359589    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5988d4cf74-28fvc" podStartSLOduration=1.646487179 podStartE2EDuration="2.359572806s" totalImagesPullingTime="713.085627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:24:42.358034654 +0000 UTC m=+813.774960999" watchObservedRunningTime="2026-07-13 21:24:42.359572806 +0000 UTC m=+813.776499155"
Jul 13 21:24:53.350791 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:24:53.350736    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5988d4cf74-28fvc"
Jul 13 21:25:26.889014 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:26.888979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:26.991433 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:26.991401    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:27.122392 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:27.122337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:32.784896 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:32.784850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:32.856721 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:32.856692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:32.968065 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:32.968034    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:33.159474 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:33.159445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:39.007250 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:39.007216    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:25:39.344444 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:25:39.344352    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:26:09.115078 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:26:09.115008    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:26:09.115078 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:26:09.115019    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:28:39.428753 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:28:39.428719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:28:56.538895 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:28:56.538859    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:30:27.031120 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:30:27.031075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:30:27.118836 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:30:27.118794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:31:09.145241 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:31:09.145200    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:31:09.146681 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:31:09.146662    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:32:22.306439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:32:22.306363    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:34:31.126382 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:34:31.126343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:35:02.673495 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:35:02.673408    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:35:02.774820 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:35:02.774775    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:09.167249 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:09.167220    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:36:09.170346 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:09.170327    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:36:23.203157 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:23.203124    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:27.608200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:27.608167    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:35.874957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:35.874873    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:40.957010 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:40.956970    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:44.014648 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:44.014614    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:44.104430 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:44.104399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:44.459915 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:44.459877    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:50.255269 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:50.255231    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:50.355212 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:50.355181    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:50.488526 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:50.488497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:50.637948 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:50.637920    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:51.660655 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:51.660613    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:51.750928 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:51.750885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:51.815933 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:51.815897    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:52.126121 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:52.126039    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:52.318152 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:52.318117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:52.418552 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:52.418520    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:36:54.212461 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:36:54.212427    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:07.144878 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:07.144840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:11.788550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:11.788512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:17.599231 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:17.599194    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:17.886397 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:17.886307    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:21.996724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:21.996688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:22.376885 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:22.376711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:22.476011 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:22.475965    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:22.596498 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:22.596459    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:23.828778 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:23.828732    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:24.009597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:24.009551    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:24.419123 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:24.419093    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:24.542798 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:24.542752    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:24.644084 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:24.644041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:28.472914 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:28.472884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:28.573152 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:28.573116    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:31.869156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:31.869122    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:38.883747 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:38.883717    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:43.428649 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:43.428615    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:46.157586 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:46.157558    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:49.239195 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:49.239150    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:49.301788 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:49.301743    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:53.626227 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:53.626190    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:53.932325 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:53.932295    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.136347 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.136305    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.232501 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.232471    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.305903 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.305865    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.401929 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.401875    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.901024 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.900988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:37:55.998692 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:37:55.998653    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:00.224202 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.224124    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"]
Jul 13 21:38:00.227356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.227335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:00.229905 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.229885    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:38:00.230027 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.229923    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:38:00.235233 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.235198    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"]
Jul 13 21:38:00.303726 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.303666    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:00.356040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.356000    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5278c205-t56tv\" (UID: \"6f1bed64-be6d-47f6-828a-c634c52dee14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:00.405534 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.405484    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:00.456475 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.456442    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5278c205-t56tv\" (UID: \"6f1bed64-be6d-47f6-828a-c634c52dee14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:00.457073 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.457045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-5278c205-t56tv\" (UID: \"6f1bed64-be6d-47f6-828a-c634c52dee14\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:00.538836 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.538724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:00.672140 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.672118    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"]
Jul 13 21:38:00.674442 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:38:00.674409    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1bed64_be6d_47f6_828a_c634c52dee14.slice/crio-bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288 WatchSource:0}: Error finding container bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288: Status 404 returned error can't find the container with id bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288
Jul 13 21:38:00.677216 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:00.677199    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:38:01.645705 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:01.645665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv" event={"ID":"6f1bed64-be6d-47f6-828a-c634c52dee14","Type":"ContainerStarted","Data":"bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288"}
Jul 13 21:38:03.656081 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:03.656043    2577 generic.go:356] "Generic (PLEG): container finished" podID="6f1bed64-be6d-47f6-828a-c634c52dee14" containerID="03e1b751845aa01a487e6dea904d530f672a2fa2a40d911f36ba072618135e32" exitCode=0
Jul 13 21:38:03.656457 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:03.656139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv" event={"ID":"6f1bed64-be6d-47f6-828a-c634c52dee14","Type":"ContainerDied","Data":"03e1b751845aa01a487e6dea904d530f672a2fa2a40d911f36ba072618135e32"}
Jul 13 21:38:04.025124 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:04.025090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:04.793590 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:04.793564    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:04.893592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:04.893548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca\") pod \"6f1bed64-be6d-47f6-828a-c634c52dee14\" (UID: \"6f1bed64-be6d-47f6-828a-c634c52dee14\") "
Jul 13 21:38:04.893950 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:04.893927    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f1bed64-be6d-47f6-828a-c634c52dee14" (UID: "6f1bed64-be6d-47f6-828a-c634c52dee14"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:38:04.994200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:04.994098    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1bed64-be6d-47f6-828a-c634c52dee14-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:38:05.657349 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:05.657320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:05.666746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:05.666723    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv"
Jul 13 21:38:05.666746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:05.666733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-5278c205-t56tv" event={"ID":"6f1bed64-be6d-47f6-828a-c634c52dee14","Type":"ContainerDied","Data":"bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288"}
Jul 13 21:38:05.666921 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:05.666797    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bba7c998cef61b87afc120a830c734c1757188b97aefb914a16f718dbc75a288"
Jul 13 21:38:10.823645 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:10.823606    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:15.311527 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:15.311490    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:21.147169 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:21.147132    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:21.244926 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:21.244891    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:26.379122 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:26.379085    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:26.686001 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:26.685958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:27.905609 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:27.905577    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:27.997819 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:27.997791    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:28.067295 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:28.067263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:28.168025 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:28.167941    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:28.643935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:28.643905    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:28.744475 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:28.744437    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:32.201142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.201106    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:32.201537 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.201429    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f1bed64-be6d-47f6-828a-c634c52dee14" containerName="revoke-keys"
Jul 13 21:38:32.201537 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.201445    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f1bed64-be6d-47f6-828a-c634c52dee14" containerName="revoke-keys"
Jul 13 21:38:32.201537 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.201504    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f1bed64-be6d-47f6-828a-c634c52dee14" containerName="revoke-keys"
Jul 13 21:38:32.204433 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.204415    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:32.207143 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.207119    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:38:32.207143 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.207129    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:38:32.210738 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.210715    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:32.324364 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.324325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw\" (UID: \"0ede814f-6fef-4a13-9e1a-4739fb0d4d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:32.425597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.425559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw\" (UID: \"0ede814f-6fef-4a13-9e1a-4739fb0d4d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:32.426223 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.426193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw\" (UID: \"0ede814f-6fef-4a13-9e1a-4739fb0d4d47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:32.516589 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.516485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:32.654385 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.652142    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:32.659002 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:38:32.658974    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ede814f_6fef_4a13_9e1a_4739fb0d4d47.slice/crio-15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e WatchSource:0}: Error finding container 15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e: Status 404 returned error can't find the container with id 15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e
Jul 13 21:38:32.785558 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.785516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" event={"ID":"0ede814f-6fef-4a13-9e1a-4739fb0d4d47","Type":"ContainerStarted","Data":"15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e"}
Jul 13 21:38:32.797121 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.797090    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:32.898088 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:32.898062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:33.791061 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:33.791024    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerID="0eca2411437fa9c60662da70b6966c9425c2436e689c3a63079f7206380fd58b" exitCode=7
Jul 13 21:38:33.791546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:33.791090    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" event={"ID":"0ede814f-6fef-4a13-9e1a-4739fb0d4d47","Type":"ContainerDied","Data":"0eca2411437fa9c60662da70b6966c9425c2436e689c3a63079f7206380fd58b"}
Jul 13 21:38:33.791546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:33.791346    2577 scope.go:117] "RemoveContainer" containerID="0eca2411437fa9c60662da70b6966c9425c2436e689c3a63079f7206380fd58b"
Jul 13 21:38:34.797890 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:34.797811    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerID="a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9" exitCode=7
Jul 13 21:38:34.798232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:34.797894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" event={"ID":"0ede814f-6fef-4a13-9e1a-4739fb0d4d47","Type":"ContainerDied","Data":"a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9"}
Jul 13 21:38:34.798232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:34.797957    2577 scope.go:117] "RemoveContainer" containerID="0eca2411437fa9c60662da70b6966c9425c2436e689c3a63079f7206380fd58b"
Jul 13 21:38:34.798232 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:34.798151    2577 scope.go:117] "RemoveContainer" containerID="a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9"
Jul 13 21:38:34.798376 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:38:34.798348    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw_odh-ai-gateway-infra(0ede814f-6fef-4a13-9e1a-4739fb0d4d47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47"
Jul 13 21:38:35.804054 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:35.804027    2577 scope.go:117] "RemoveContainer" containerID="a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9"
Jul 13 21:38:35.804419 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:38:35.804232    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw_odh-ai-gateway-infra(0ede814f-6fef-4a13-9e1a-4739fb0d4d47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47"
Jul 13 21:38:36.503912 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:36.503884    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:38:48.177757 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:48.177726    2577 scope.go:117] "RemoveContainer" containerID="a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9"
Jul 13 21:38:48.866130 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:48.866094    2577 generic.go:356] "Generic (PLEG): container finished" podID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerID="43b32672890f5b3c1c52e514990e2751ef8545f3c534d997e464a2937ffabab9" exitCode=0
Jul 13 21:38:48.866283 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:48.866164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" event={"ID":"0ede814f-6fef-4a13-9e1a-4739fb0d4d47","Type":"ContainerDied","Data":"43b32672890f5b3c1c52e514990e2751ef8545f3c534d997e464a2937ffabab9"}
Jul 13 21:38:48.866283 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:48.866253    2577 scope.go:117] "RemoveContainer" containerID="a38c68872d3e1d9381a9ec03df5bd466906ef23b1179edcb07acffc6af64faf9"
Jul 13 21:38:49.892425 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:49.892379    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:50.004027 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.004000    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:50.083897 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.083857    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca\") pod \"0ede814f-6fef-4a13-9e1a-4739fb0d4d47\" (UID: \"0ede814f-6fef-4a13-9e1a-4739fb0d4d47\") "
Jul 13 21:38:50.084235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.084209    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0ede814f-6fef-4a13-9e1a-4739fb0d4d47" (UID: "0ede814f-6fef-4a13-9e1a-4739fb0d4d47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:38:50.184425 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.184399    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0ede814f-6fef-4a13-9e1a-4739fb0d4d47-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:38:50.878101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.878071    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"
Jul 13 21:38:50.878101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.878083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw" event={"ID":"0ede814f-6fef-4a13-9e1a-4739fb0d4d47","Type":"ContainerDied","Data":"15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e"}
Jul 13 21:38:50.878291 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.878133    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15ca873d94402e3e3950eb0628de5990fe53416aaa9db264033a792adefdbb6e"
Jul 13 21:38:50.906138 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.906105    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:50.911501 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:50.911475    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-k4fbw"]
Jul 13 21:38:51.182129 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:38:51.182090    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" path="/var/lib/kubelet/pods/0ede814f-6fef-4a13-9e1a-4739fb0d4d47/volumes"
Jul 13 21:39:22.250553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250512    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"]
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250845    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250858    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250879    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250885    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250891    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250896    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250960    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.251108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.250967    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:39:22.253717 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.253700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:22.256626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.256600    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:39:22.256626 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.256619    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:39:22.262243 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.262215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"]
Jul 13 21:39:22.343392 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.343361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr\" (UID: \"21538795-896a-4abb-ba61-200ad4f5c3c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:22.444679 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.444642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr\" (UID: \"21538795-896a-4abb-ba61-200ad4f5c3c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:22.445345 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.445322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr\" (UID: \"21538795-896a-4abb-ba61-200ad4f5c3c5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:22.565666 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.565582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:22.697738 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:22.697711    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"]
Jul 13 21:39:22.700238 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:39:22.700209    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21538795_896a_4abb_ba61_200ad4f5c3c5.slice/crio-8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6 WatchSource:0}: Error finding container 8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6: Status 404 returned error can't find the container with id 8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6
Jul 13 21:39:23.016453 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:23.016420    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr" event={"ID":"21538795-896a-4abb-ba61-200ad4f5c3c5","Type":"ContainerStarted","Data":"8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6"}
Jul 13 21:39:24.021747 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:24.021712    2577 generic.go:356] "Generic (PLEG): container finished" podID="21538795-896a-4abb-ba61-200ad4f5c3c5" containerID="ab1eac6b55a1828e051f4e6d3aa6e703ec76b622ffcdede62bdc62a67b4bc23a" exitCode=0
Jul 13 21:39:24.022210 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:24.021754    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr" event={"ID":"21538795-896a-4abb-ba61-200ad4f5c3c5","Type":"ContainerDied","Data":"ab1eac6b55a1828e051f4e6d3aa6e703ec76b622ffcdede62bdc62a67b4bc23a"}
Jul 13 21:39:25.152390 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:25.152363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:25.267944 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:25.267909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca\") pod \"21538795-896a-4abb-ba61-200ad4f5c3c5\" (UID: \"21538795-896a-4abb-ba61-200ad4f5c3c5\") "
Jul 13 21:39:25.268314 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:25.268289    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21538795-896a-4abb-ba61-200ad4f5c3c5" (UID: "21538795-896a-4abb-ba61-200ad4f5c3c5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:39:25.368532 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:25.368445    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21538795-896a-4abb-ba61-200ad4f5c3c5-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:39:26.031591 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:26.031552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr" event={"ID":"21538795-896a-4abb-ba61-200ad4f5c3c5","Type":"ContainerDied","Data":"8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6"}
Jul 13 21:39:26.031799 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:26.031608    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8c70d405c7e64a56545a76203fd9fab0fa3a6715e2568b312004a1d6abad66d6"
Jul 13 21:39:26.031799 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:26.031563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr"
Jul 13 21:39:33.062030 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:33.061930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:37.346748 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:37.346710    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:43.286988 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:43.286951    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:43.386185 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:43.386149    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:47.542686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:47.542647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:48.200016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:48.199985    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:48.340208 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:48.340179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:49.793783 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:49.793725    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:53.908608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:53.908578    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:55.936965 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:55.936930    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:56.054536 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:56.054493    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:56.167632 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:56.167596    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:56.282984 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:56.282898    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:39:56.701198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:39:56.701165    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:04.014173 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:04.014125    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:04.115137 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:04.115104    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:22.701643 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:22.701607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:28.436828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:28.436787    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:28.539394 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:28.539359    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:33.613378 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:33.613337    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:33.711324 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:33.711292    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:49.572729 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:49.572688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:55.620401 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:55.620370    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:55.681724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:55.681692    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:55.750980 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:55.750912    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:40:55.848314 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:40:55.848240    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:06.442944 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:06.442909    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:07.315686 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:07.315647    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:09.192076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:09.192051    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:41:09.196072 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:09.196050    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:41:13.112877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:13.112845    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:13.196107 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:13.196075    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:13.274793 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:13.274744    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:13.374496 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:13.374411    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:13.473899 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:13.473862    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:14.300778 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:14.300730    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:14.399921 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:14.399883    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:14.500173 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:14.500128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:16.201228 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:16.201195    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:21.950154 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:21.950108    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:22.014026 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:22.013988    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:22.692589 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:22.692555    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:27.248242 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:27.248207    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:27.327388 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:27.327358    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:28.533587 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:28.533553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:28.626426 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:28.626396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:28.770330 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:28.770303    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:29.082800 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:29.082750    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:29.202404 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:29.202376    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:31.978831 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:31.978794    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:32.146095 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:32.146060    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:32.276115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:32.276032    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:32.722680 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:32.722651    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:32.823328 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:32.823298    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:36.514161 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:36.514131    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:37.218054 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:37.218013    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:37.316637 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:37.316609    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:37.731250 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:37.731220    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:43.877648 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:43.877604    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:43.978989 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:43.978957    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:44.240167 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.240128    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"]
Jul 13 21:41:44.240452 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.240440    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21538795-896a-4abb-ba61-200ad4f5c3c5" containerName="revoke-keys"
Jul 13 21:41:44.240503 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.240454    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="21538795-896a-4abb-ba61-200ad4f5c3c5" containerName="revoke-keys"
Jul 13 21:41:44.240540 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.240525    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ede814f-6fef-4a13-9e1a-4739fb0d4d47" containerName="revoke-keys"
Jul 13 21:41:44.240540 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.240536    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="21538795-896a-4abb-ba61-200ad4f5c3c5" containerName="revoke-keys"
Jul 13 21:41:44.243633 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.243610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:44.246033 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.246011    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:41:44.246153 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.246035    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:41:44.250553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.250525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"]
Jul 13 21:41:44.266082 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.266048    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl\" (UID: \"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:44.367048 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.367007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl\" (UID: \"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:44.367666 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.367641    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl\" (UID: \"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:44.554960 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.554868    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:44.693460 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:44.693418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"]
Jul 13 21:41:44.696706 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:41:44.696668    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc907c10a_7287_4cf6_a69e_ae3e03f5d9b0.slice/crio-1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6 WatchSource:0}: Error finding container 1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6: Status 404 returned error can't find the container with id 1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6
Jul 13 21:41:45.612540 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:45.612502    2577 generic.go:356] "Generic (PLEG): container finished" podID="c907c10a-7287-4cf6-a69e-ae3e03f5d9b0" containerID="447f9950020e4843ec0b701d33e36d512a4338917e3817cb2734f1489d06fae7" exitCode=0
Jul 13 21:41:45.612949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:45.612566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl" event={"ID":"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0","Type":"ContainerDied","Data":"447f9950020e4843ec0b701d33e36d512a4338917e3817cb2734f1489d06fae7"}
Jul 13 21:41:45.612949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:45.612630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl" event={"ID":"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0","Type":"ContainerStarted","Data":"1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6"}
Jul 13 21:41:46.742345 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:46.742315    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:46.785491 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:46.785459    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca\") pod \"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0\" (UID: \"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0\") "
Jul 13 21:41:46.785876 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:46.785853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c907c10a-7287-4cf6-a69e-ae3e03f5d9b0" (UID: "c907c10a-7287-4cf6-a69e-ae3e03f5d9b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:41:46.886227 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:46.886139    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c907c10a-7287-4cf6-a69e-ae3e03f5d9b0-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:41:47.623991 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:47.623907    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl"
Jul 13 21:41:47.624230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:47.623913    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl" event={"ID":"c907c10a-7287-4cf6-a69e-ae3e03f5d9b0","Type":"ContainerDied","Data":"1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6"}
Jul 13 21:41:47.624230 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:47.624040    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1d8fc5a208520b936289658cd2837c7a865687e8dc11ca9cb67b61472c7119c6"
Jul 13 21:41:54.747442 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:54.747407    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:41:59.328680 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:41:59.328646    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:05.191676 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:05.191641    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:05.291903 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:05.291866    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:10.009644 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:10.009616    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:10.242536 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:10.242503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:10.372906 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:10.372823    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:10.473728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:10.473688    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:10.589477 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:10.589445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:16.423480 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:16.423449    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:16.527750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:16.527719    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:16.623724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:16.623682    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:16.771630 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:16.771542    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:22.379045 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:22.378931    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:22.449779 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:22.449728    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:29.193456 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:29.193424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:29.518062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:29.517978    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:29.820355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:29.820260    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.010550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.010512    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.082474 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.082424    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.177391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.177347    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.377657 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.377566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.695160 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.695126    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:32.871865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:32.871834    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:36.574476 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:36.574445    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:46.152924 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.152892    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:42:46.153322 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.153218    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c907c10a-7287-4cf6-a69e-ae3e03f5d9b0" containerName="revoke-keys"
Jul 13 21:42:46.153322 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.153229    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c907c10a-7287-4cf6-a69e-ae3e03f5d9b0" containerName="revoke-keys"
Jul 13 21:42:46.153322 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.153295    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c907c10a-7287-4cf6-a69e-ae3e03f5d9b0" containerName="revoke-keys"
Jul 13 21:42:46.156262 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.156240    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.159958 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.159932    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffa5db90\"/\"default-dockercfg-j2fgk\""
Jul 13 21:42:46.160105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.159965    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffa5db90\"/\"openshift-service-ca.crt\""
Jul 13 21:42:46.160105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.159987    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ffa5db90\"/\"e2e-lifecycle-model-ffa5db90-kserve-self-signed-certs\""
Jul 13 21:42:46.160105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.160057    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ffa5db90\"/\"kube-root-ca.crt\""
Jul 13 21:42:46.168537 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.168510    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:42:46.199349 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.199320    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:42:46.200366 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.200493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.200493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnbd2\" (UniqueName: \"kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.200493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.200493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.200493 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.200481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301062 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301066    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301144    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnbd2\" (UniqueName: \"kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301521 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301615 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.301615 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.301604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.303470 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.303447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.303596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.303578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.309244 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.309225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnbd2\" (UniqueName: \"kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2\") pod \"e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") " pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.466273 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.466180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:46.593229 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.593203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:42:46.596064 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:42:46.596038    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda64cebc6_ebcc_406d_b9ea_20262b55d0fa.slice/crio-820ca5d6b80ae96d3591107397bb0df9b49f9b42a0479f29db7f93c2c603c5ba WatchSource:0}: Error finding container 820ca5d6b80ae96d3591107397bb0df9b49f9b42a0479f29db7f93c2c603c5ba: Status 404 returned error can't find the container with id 820ca5d6b80ae96d3591107397bb0df9b49f9b42a0479f29db7f93c2c603c5ba
Jul 13 21:42:46.881389 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.881285    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" event={"ID":"a64cebc6-ebcc-406d-b9ea-20262b55d0fa","Type":"ContainerStarted","Data":"773a7fd41e4ac74af2b7cbdab455383ea8d556ae22e5332861f7a025720de26c"}
Jul 13 21:42:46.881389 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.881336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" event={"ID":"a64cebc6-ebcc-406d-b9ea-20262b55d0fa","Type":"ContainerStarted","Data":"820ca5d6b80ae96d3591107397bb0df9b49f9b42a0479f29db7f93c2c603c5ba"}
Jul 13 21:42:46.897782 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:46.897721    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" podStartSLOduration=0.897707905 podStartE2EDuration="897.707905ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:46.897235082 +0000 UTC m=+1898.314161426" watchObservedRunningTime="2026-07-13 21:42:46.897707905 +0000 UTC m=+1898.314634248"
Jul 13 21:42:56.466404 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:56.466361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:56.466928 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:56.466422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:56.478740 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:56.478712    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:42:56.933714 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:42:56.933686    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:43:25.392927 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:25.392886    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:43:30.023260 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:30.023227    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:43:35.005042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:35.005007    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:43:35.049100 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:35.049059    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:43:50.025267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.025206    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" podUID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" containerName="main" containerID="cri-o://773a7fd41e4ac74af2b7cbdab455383ea8d556ae22e5332861f7a025720de26c" gracePeriod=45
Jul 13 21:43:50.146369 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.146339    2577 generic.go:356] "Generic (PLEG): container finished" podID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" containerID="773a7fd41e4ac74af2b7cbdab455383ea8d556ae22e5332861f7a025720de26c" exitCode=0
Jul 13 21:43:50.146494 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.146432    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" event={"ID":"a64cebc6-ebcc-406d-b9ea-20262b55d0fa","Type":"ContainerDied","Data":"773a7fd41e4ac74af2b7cbdab455383ea8d556ae22e5332861f7a025720de26c"}
Jul 13 21:43:50.269095 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.269062    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:43:50.351334 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351235    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351334 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351334 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351321    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rnbd2\" (UniqueName: \"kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351387    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351439    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir\") pod \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\" (UID: \"a64cebc6-ebcc-406d-b9ea-20262b55d0fa\") "
Jul 13 21:43:50.351623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351548    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home" (OuterVolumeSpecName: "home") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:50.351877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351635    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:50.351877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351785    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:50.351877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351810    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-home\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:50.351877 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.351831    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-model-cache\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:50.353652 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.353631    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:43:50.353652 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.353645    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm" (OuterVolumeSpecName: "dshm") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:43:50.353774 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.353660    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2" (OuterVolumeSpecName: "kube-api-access-rnbd2") pod "a64cebc6-ebcc-406d-b9ea-20262b55d0fa" (UID: "a64cebc6-ebcc-406d-b9ea-20262b55d0fa"). InnerVolumeSpecName "kube-api-access-rnbd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:43:50.452604 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.452555    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-dshm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:50.452604 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.452598    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnbd2\" (UniqueName: \"kubernetes.io/projected/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-kube-api-access-rnbd2\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:50.452604 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.452609    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tls-certs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:50.452604 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:50.452618    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a64cebc6-ebcc-406d-b9ea-20262b55d0fa-tmp-dir\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:43:51.152279 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.152251    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"
Jul 13 21:43:51.152727 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.152276    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs" event={"ID":"a64cebc6-ebcc-406d-b9ea-20262b55d0fa","Type":"ContainerDied","Data":"820ca5d6b80ae96d3591107397bb0df9b49f9b42a0479f29db7f93c2c603c5ba"}
Jul 13 21:43:51.152727 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.152328    2577 scope.go:117] "RemoveContainer" containerID="773a7fd41e4ac74af2b7cbdab455383ea8d556ae22e5332861f7a025720de26c"
Jul 13 21:43:51.173219 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.173191    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:43:51.174608 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.174584    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ffa5db90/e2e-lifecycle-model-ffa5db90-kserve-6c6f6dc7c9-4l6zs"]
Jul 13 21:43:51.182238 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:43:51.182213    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" path="/var/lib/kubelet/pods/a64cebc6-ebcc-406d-b9ea-20262b55d0fa/volumes"
Jul 13 21:44:11.980883 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:11.980840    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:12.114145 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:12.114115    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:17.570057 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:17.570023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:17.630222 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:17.630184    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:22.377945 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:22.377919    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:22.630582 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:22.630503    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:23.728142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:23.728101    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:23.825337 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:23.825306    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:23.927078 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:23.927048    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:24.657145 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:24.657112    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:24.770358 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:24.770326    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:24.867300 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:24.867272    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:26.395676 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:26.395634    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:32.051156 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:32.051121    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:32.161946 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:32.161916    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:32.760985 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:32.760953    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:38.604358 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:38.604329    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:39.853724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:39.853687    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:49.339515 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:49.339476    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:44:49.690211 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:44:49.690169    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:00.140229 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.140191    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"]
Jul 13 21:45:00.140594 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.140522    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" containerName="main"
Jul 13 21:45:00.140594 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.140533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" containerName="main"
Jul 13 21:45:00.140594 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.140594    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a64cebc6-ebcc-406d-b9ea-20262b55d0fa" containerName="main"
Jul 13 21:45:00.143582 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.143566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"
Jul 13 21:45:00.145956 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.145933    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:45:00.149577 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.149545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"]
Jul 13 21:45:00.154673 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.154648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"
Jul 13 21:45:00.278486 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.278462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"]
Jul 13 21:45:00.281080 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:45:00.281051    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c05ee3a_9be4_44d7_8cb6_027d2dcd7d8b.slice/crio-aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27 WatchSource:0}: Error finding container aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27: Status 404 returned error can't find the container with id aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27
Jul 13 21:45:00.452895 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:00.452804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75" event={"ID":"7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b","Type":"ContainerStarted","Data":"aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27"}
Jul 13 21:45:01.458563 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:01.458527    2577 generic.go:356] "Generic (PLEG): container finished" podID="7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b" containerID="736ccc0632a85c184fb5441cf7f40a42b88f784bbbb2e603ef62ac0a7144cc14" exitCode=0
Jul 13 21:45:01.458975 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:01.458607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75" event={"ID":"7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b","Type":"ContainerDied","Data":"736ccc0632a85c184fb5441cf7f40a42b88f784bbbb2e603ef62ac0a7144cc14"}
Jul 13 21:45:02.587868 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:02.587840    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"
Jul 13 21:45:03.468142 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:03.468069    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75"
Jul 13 21:45:03.468275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:03.468069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732985-znj75" event={"ID":"7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b","Type":"ContainerDied","Data":"aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27"}
Jul 13 21:45:03.468275 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:03.468201    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aabf7c8481ecea1d37100f6264cf24d8cae0122779d5596dfedf7bde50be9f27"
Jul 13 21:45:03.933079 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:03.933040    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:04.996735 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:04.996694    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:05.092088 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:05.092049    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:09.260318 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:09.260288    2577 scope.go:117] "RemoveContainer" containerID="43b32672890f5b3c1c52e514990e2751ef8545f3c534d997e464a2937ffabab9"
Jul 13 21:45:10.681106 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:10.681076    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:16.612063 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:16.612029    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:16.713139 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:16.713105    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:17.069616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:17.069585    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:17.312938 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:17.312908    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:17.435773 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:17.435739    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:19.941108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:19.941077    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:20.038698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:20.038667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:20.252531 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:20.252448    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:20.473463 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:20.473434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:20.753490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:20.753454    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:20.855378 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:20.855343    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:24.483098 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:24.483062    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:24.831069 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:24.830991    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:25.181887 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:25.181850    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:25.680025 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:25.679996    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.010074 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.010041    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.104389 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.104346    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.169428 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.169399    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.558518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.558487    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.768335 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.768263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:28.867714 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:28.867599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:32.518511 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:32.518430    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:42.322363 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:42.322327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:42.416984 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:42.416944    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:47.512469 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:47.512426    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:50.559589 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:50.559560    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:45:50.661168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:45:50.661113    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:09.215329 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:09.215301    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:46:09.220991 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:09.220969    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:46:12.358749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.358717    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:12.359291 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.359164    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b" containerName="cleanup"
Jul 13 21:46:12.359291 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.359181    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b" containerName="cleanup"
Jul 13 21:46:12.359291 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.359271    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b" containerName="cleanup"
Jul 13 21:46:12.362101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.362081    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.364675 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.364650    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f491fd\"/\"test-model-f491fd-kserve-self-signed-certs\""
Jul 13 21:46:12.365754 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.365732    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f491fd\"/\"openshift-service-ca.crt\""
Jul 13 21:46:12.365928 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.365825    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f491fd\"/\"kube-root-ca.crt\""
Jul 13 21:46:12.365928 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.365843    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f491fd\"/\"default-dockercfg-4qhbp\""
Jul 13 21:46:12.368309 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368287    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.368412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twzdf\" (UniqueName: \"kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.368412 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.368487 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.368532 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.368532 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.368520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.371844 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.371819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:12.400462 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.400433    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:12.469333 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469300    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469333 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twzdf\" (UniqueName: \"kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469441    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469788 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469788 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469781    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.469912 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.469820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.471701 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.471666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.472008 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.471987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.477646 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.477627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twzdf\" (UniqueName: \"kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf\") pod \"test-model-f491fd-kserve-64d757fb45-jzsfh\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") " pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.673121 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.673087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:12.837550 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:12.837521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:12.849945 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:46:12.849916    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae4e93af_f45a_405c_8019_6072d99edef3.slice/crio-70a2d710b2018d3f063f78c19d62e38afddab98f0b744defb5494f1f6b3b39b2 WatchSource:0}: Error finding container 70a2d710b2018d3f063f78c19d62e38afddab98f0b744defb5494f1f6b3b39b2: Status 404 returned error can't find the container with id 70a2d710b2018d3f063f78c19d62e38afddab98f0b744defb5494f1f6b3b39b2
Jul 13 21:46:13.774718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:13.774679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" event={"ID":"ae4e93af-f45a-405c-8019-6072d99edef3","Type":"ContainerStarted","Data":"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"}
Jul 13 21:46:13.774718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:13.774722    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" event={"ID":"ae4e93af-f45a-405c-8019-6072d99edef3","Type":"ContainerStarted","Data":"70a2d710b2018d3f063f78c19d62e38afddab98f0b744defb5494f1f6b3b39b2"}
Jul 13 21:46:13.791235 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:13.791173    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" podStartSLOduration=1.791153762 podStartE2EDuration="1.791153762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:46:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:46:13.790992601 +0000 UTC m=+2105.207918956" watchObservedRunningTime="2026-07-13 21:46:13.791153762 +0000 UTC m=+2105.208080106"
Jul 13 21:46:22.674033 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:22.673992    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:22.674433 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:22.674057    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:22.686011 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:22.685982    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:22.827992 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:22.827961    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:30.955977 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:30.955946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:31.938892 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:31.938862    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:31.960528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:31.960497    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:32.550682 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:32.550626    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:32.651377 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:32.651349    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:42.549052 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:42.549000    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:45.966459 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:45.966428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:46.066701 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:46.066667    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:46.958404 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:46.958363    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" podUID="ae4e93af-f45a-405c-8019-6072d99edef3" containerName="main" containerID="cri-o://f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47" gracePeriod=45
Jul 13 21:46:47.206597 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.206569    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:47.263784 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.263677    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:47.369682 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369651    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.369911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369710    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.369911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.369911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twzdf\" (UniqueName: \"kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.369911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.369911 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369833    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home\") pod \"ae4e93af-f45a-405c-8019-6072d99edef3\" (UID: \"ae4e93af-f45a-405c-8019-6072d99edef3\") "
Jul 13 21:46:47.370172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:47.370172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.369992    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:47.370172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.370154    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home" (OuterVolumeSpecName: "home") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:47.371982 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.371957    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm" (OuterVolumeSpecName: "dshm") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:47.372080 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.372027    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:46:47.372080 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.372066    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf" (OuterVolumeSpecName: "kube-api-access-twzdf") pod "ae4e93af-f45a-405c-8019-6072d99edef3" (UID: "ae4e93af-f45a-405c-8019-6072d99edef3"). InnerVolumeSpecName "kube-api-access-twzdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:46:47.470889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470837    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae4e93af-f45a-405c-8019-6072d99edef3-tls-certs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.470889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470899    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-tmp-dir\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.470889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470909    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twzdf\" (UniqueName: \"kubernetes.io/projected/ae4e93af-f45a-405c-8019-6072d99edef3-kube-api-access-twzdf\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.471146 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470918    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-dshm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.471146 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470929    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-home\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.471146 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.470936    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae4e93af-f45a-405c-8019-6072d99edef3-model-cache\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:46:47.922427 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.922396    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae4e93af-f45a-405c-8019-6072d99edef3" containerID="f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47" exitCode=0
Jul 13 21:46:47.922613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.922439    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"
Jul 13 21:46:47.922613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.922490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" event={"ID":"ae4e93af-f45a-405c-8019-6072d99edef3","Type":"ContainerDied","Data":"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"}
Jul 13 21:46:47.922613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.922537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh" event={"ID":"ae4e93af-f45a-405c-8019-6072d99edef3","Type":"ContainerDied","Data":"70a2d710b2018d3f063f78c19d62e38afddab98f0b744defb5494f1f6b3b39b2"}
Jul 13 21:46:47.922613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.922557    2577 scope.go:117] "RemoveContainer" containerID="f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"
Jul 13 21:46:47.931026 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.931006    2577 scope.go:117] "RemoveContainer" containerID="f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"
Jul 13 21:46:47.931300 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:46:47.931282    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47\": container with ID starting with f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47 not found: ID does not exist" containerID="f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"
Jul 13 21:46:47.931343 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.931314    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47"} err="failed to get container status \"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47\": rpc error: code = NotFound desc = could not find container \"f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47\": container with ID starting with f83fd14f150b19716696851727279d37c2b8a393fdd144940a9f8aed63babe47 not found: ID does not exist"
Jul 13 21:46:47.943590 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.943566    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:47.946799 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:47.946744    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f491fd/test-model-f491fd-kserve-64d757fb45-jzsfh"]
Jul 13 21:46:49.182355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:49.182321    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae4e93af-f45a-405c-8019-6072d99edef3" path="/var/lib/kubelet/pods/ae4e93af-f45a-405c-8019-6072d99edef3/volumes"
Jul 13 21:46:53.281458 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:53.281428    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:53.347411 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:53.347364    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:57.889705 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:57.889674    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:58.129976 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.129932    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:58.238449 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.238370    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"]
Jul 13 21:46:58.238679 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.238668    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae4e93af-f45a-405c-8019-6072d99edef3" containerName="main"
Jul 13 21:46:58.238721 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.238680    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae4e93af-f45a-405c-8019-6072d99edef3" containerName="main"
Jul 13 21:46:58.238755 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.238741    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae4e93af-f45a-405c-8019-6072d99edef3" containerName="main"
Jul 13 21:46:58.242518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.242498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:46:58.244941 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.244919    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:46:58.245068 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.244968    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:46:58.248125 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.248100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"]
Jul 13 21:46:58.365819 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.365789    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp\" (UID: \"668ae9e0-581e-4a25-b2c1-dde541b5a4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:46:58.467056 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.467020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp\" (UID: \"668ae9e0-581e-4a25-b2c1-dde541b5a4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:46:58.467650 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.467631    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp\" (UID: \"668ae9e0-581e-4a25-b2c1-dde541b5a4cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:46:58.554216 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.554131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:46:58.680355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.680326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"]
Jul 13 21:46:58.683070 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:46:58.683032    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod668ae9e0_581e_4a25_b2c1_dde541b5a4cc.slice/crio-cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909 WatchSource:0}: Error finding container cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909: Status 404 returned error can't find the container with id cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909
Jul 13 21:46:58.848243 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.848213    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:58.954906 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.954874    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:46:58.970935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.970903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp" event={"ID":"668ae9e0-581e-4a25-b2c1-dde541b5a4cc","Type":"ContainerStarted","Data":"81e15a092908c572e701e84743d02813d3397f5fd23754272a3f65b335f5e111"}
Jul 13 21:46:58.971084 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.970987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp" event={"ID":"668ae9e0-581e-4a25-b2c1-dde541b5a4cc","Type":"ContainerStarted","Data":"cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909"}
Jul 13 21:46:58.986874 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:58.986822    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp" podStartSLOduration=0.986802733 podStartE2EDuration="986.802733ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:46:58.985560086 +0000 UTC m=+2150.402486428" watchObservedRunningTime="2026-07-13 21:46:58.986802733 +0000 UTC m=+2150.403729079"
Jul 13 21:46:59.976462 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:59.976422    2577 generic.go:356] "Generic (PLEG): container finished" podID="668ae9e0-581e-4a25-b2c1-dde541b5a4cc" containerID="81e15a092908c572e701e84743d02813d3397f5fd23754272a3f65b335f5e111" exitCode=0
Jul 13 21:46:59.976862 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:46:59.976509    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp" event={"ID":"668ae9e0-581e-4a25-b2c1-dde541b5a4cc","Type":"ContainerDied","Data":"81e15a092908c572e701e84743d02813d3397f5fd23754272a3f65b335f5e111"}
Jul 13 21:47:01.111386 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.111356    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:47:01.192446 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.192413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca\") pod \"668ae9e0-581e-4a25-b2c1-dde541b5a4cc\" (UID: \"668ae9e0-581e-4a25-b2c1-dde541b5a4cc\") "
Jul 13 21:47:01.192810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.192752    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "668ae9e0-581e-4a25-b2c1-dde541b5a4cc" (UID: "668ae9e0-581e-4a25-b2c1-dde541b5a4cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:47:01.293611 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.293518    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/668ae9e0-581e-4a25-b2c1-dde541b5a4cc-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:47:01.985929 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.985901    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp"
Jul 13 21:47:01.986128 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.985895    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp" event={"ID":"668ae9e0-581e-4a25-b2c1-dde541b5a4cc","Type":"ContainerDied","Data":"cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909"}
Jul 13 21:47:01.986128 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:01.986028    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb51728d163493a58ecb82eb5624956483faa7b4cdc76db61f2065c17a411909"
Jul 13 21:47:08.687262 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:08.687232    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:11.958329 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:11.958294    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:13.326508 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:13.326469    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:19.259947 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:19.259907    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:19.335941 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:19.335903    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:24.029801 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:24.029736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:24.253310 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:24.253274    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:24.604822 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:24.604781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:24.936295 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:24.936253    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:25.514698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:25.514664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:27.642810 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:27.642734    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:27.755545 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:27.755506    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:27.909306 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:27.909229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:28.051409 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:28.051377    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:28.507553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:28.507516    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:28.610299 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:28.610270    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:32.157168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:32.157130    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:32.214528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:32.214495    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:32.605393 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:32.605302    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:32.921058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:32.921023    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:33.105653 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:33.105610    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:35.721821 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:35.721781    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:35.823638 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:35.823607    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:36.016746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:36.016654    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:36.222561 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:36.222527    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:36.561974 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:36.561942    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:36.665553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:36.665521    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:40.239937 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:40.239900    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:48.373990 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.373954    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"]
Jul 13 21:47:48.374391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.374282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="668ae9e0-581e-4a25-b2c1-dde541b5a4cc" containerName="revoke-keys"
Jul 13 21:47:48.374391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.374296    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="668ae9e0-581e-4a25-b2c1-dde541b5a4cc" containerName="revoke-keys"
Jul 13 21:47:48.374391 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.374358    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="668ae9e0-581e-4a25-b2c1-dde541b5a4cc" containerName="revoke-keys"
Jul 13 21:47:48.377215 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.377192    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:48.379623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.379600    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:47:48.379623 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.379601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:47:48.383981 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.383959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"]
Jul 13 21:47:48.503518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.503479    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-459a1d-jf84v\" (UID: \"337f4fc1-837a-4c31-a505-fd9e662dab05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:48.604102 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.604056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-459a1d-jf84v\" (UID: \"337f4fc1-837a-4c31-a505-fd9e662dab05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:48.604727 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.604704    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-459a1d-jf84v\" (UID: \"337f4fc1-837a-4c31-a505-fd9e662dab05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:48.689087 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.689046    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:48.816314 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:48.816294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"]
Jul 13 21:47:48.818991 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:47:48.818957    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod337f4fc1_837a_4c31_a505_fd9e662dab05.slice/crio-917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e WatchSource:0}: Error finding container 917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e: Status 404 returned error can't find the container with id 917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e
Jul 13 21:47:49.000720 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:49.000638    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:49.100957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:49.100925    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:49.194058 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:49.194024    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v" event={"ID":"337f4fc1-837a-4c31-a505-fd9e662dab05","Type":"ContainerStarted","Data":"3cd563418520fbc1891e86d23f93ba30687e0e13e3ff6699124b44802f686c1b"}
Jul 13 21:47:49.194226 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:49.194078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v" event={"ID":"337f4fc1-837a-4c31-a505-fd9e662dab05","Type":"ContainerStarted","Data":"917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e"}
Jul 13 21:47:49.209559 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:49.209502    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v" podStartSLOduration=1.209487478 podStartE2EDuration="1.209487478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:47:49.208123497 +0000 UTC m=+2200.625049842" watchObservedRunningTime="2026-07-13 21:47:49.209487478 +0000 UTC m=+2200.626413822"
Jul 13 21:47:50.200459 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:50.200423    2577 generic.go:356] "Generic (PLEG): container finished" podID="337f4fc1-837a-4c31-a505-fd9e662dab05" containerID="3cd563418520fbc1891e86d23f93ba30687e0e13e3ff6699124b44802f686c1b" exitCode=0
Jul 13 21:47:50.200960 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:50.200508    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v" event={"ID":"337f4fc1-837a-4c31-a505-fd9e662dab05","Type":"ContainerDied","Data":"3cd563418520fbc1891e86d23f93ba30687e0e13e3ff6699124b44802f686c1b"}
Jul 13 21:47:51.330554 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:51.330527    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:51.425666 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:51.425633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca\") pod \"337f4fc1-837a-4c31-a505-fd9e662dab05\" (UID: \"337f4fc1-837a-4c31-a505-fd9e662dab05\") "
Jul 13 21:47:51.426012 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:51.425989    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "337f4fc1-837a-4c31-a505-fd9e662dab05" (UID: "337f4fc1-837a-4c31-a505-fd9e662dab05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:47:51.526408 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:51.526321    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/337f4fc1-837a-4c31-a505-fd9e662dab05-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:47:52.210398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:52.210369    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v"
Jul 13 21:47:52.210579 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:52.210361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-459a1d-jf84v" event={"ID":"337f4fc1-837a-4c31-a505-fd9e662dab05","Type":"ContainerDied","Data":"917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e"}
Jul 13 21:47:52.210579 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:52.210490    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="917655df534b4bdb750392a86c9e600c9a12c6d7b775ceace82855873e5a474e"
Jul 13 21:47:57.507000 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:57.506966    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:57.611218 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:57.611182    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:59.039813 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:59.039742    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:59.142910 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:59.142879    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:47:59.350735 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:47:59.350659    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:03.462980 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:03.462946    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:09.158048 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:09.158002    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:09.256158 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:09.256117    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:14.180302 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.180263    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:14.401989 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.401958    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:14.521486 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.521406    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"]
Jul 13 21:48:14.521731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.521718    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="337f4fc1-837a-4c31-a505-fd9e662dab05" containerName="revoke-keys"
Jul 13 21:48:14.521797 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.521733    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="337f4fc1-837a-4c31-a505-fd9e662dab05" containerName="revoke-keys"
Jul 13 21:48:14.521837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.521820    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="337f4fc1-837a-4c31-a505-fd9e662dab05" containerName="revoke-keys"
Jul 13 21:48:14.523995 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.523979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:14.526540 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.526515    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:48:14.526540 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.526525    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:48:14.532259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.532225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"]
Jul 13 21:48:14.615960 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.615924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc\" (UID: \"2337d12a-2317-4847-bbe4-164ed1851846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:14.717219 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.717177    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc\" (UID: \"2337d12a-2317-4847-bbe4-164ed1851846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:14.717875 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.717852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc\" (UID: \"2337d12a-2317-4847-bbe4-164ed1851846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:14.835553 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.835458    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:14.968430 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:14.968405    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"]
Jul 13 21:48:14.971013 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:48:14.970977    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2337d12a_2317_4847_bbe4_164ed1851846.slice/crio-824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20 WatchSource:0}: Error finding container 824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20: Status 404 returned error can't find the container with id 824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20
Jul 13 21:48:15.081260 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:15.081224    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:15.182510 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:15.182474    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:15.314864 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:15.314823    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc" event={"ID":"2337d12a-2317-4847-bbe4-164ed1851846","Type":"ContainerStarted","Data":"fe5392d8ad78a47f022a3a3496abde3e7e4e95477339975a4fab210edac3bc4c"}
Jul 13 21:48:15.315047 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:15.314894    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc" event={"ID":"2337d12a-2317-4847-bbe4-164ed1851846","Type":"ContainerStarted","Data":"824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20"}
Jul 13 21:48:15.330101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:15.330042    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc" podStartSLOduration=1.330023745 podStartE2EDuration="1.330023745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:48:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:48:15.328156603 +0000 UTC m=+2226.745082948" watchObservedRunningTime="2026-07-13 21:48:15.330023745 +0000 UTC m=+2226.746950089"
Jul 13 21:48:16.320515 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:16.320477    2577 generic.go:356] "Generic (PLEG): container finished" podID="2337d12a-2317-4847-bbe4-164ed1851846" containerID="fe5392d8ad78a47f022a3a3496abde3e7e4e95477339975a4fab210edac3bc4c" exitCode=0
Jul 13 21:48:16.320515 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:16.320515    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc" event={"ID":"2337d12a-2317-4847-bbe4-164ed1851846","Type":"ContainerDied","Data":"fe5392d8ad78a47f022a3a3496abde3e7e4e95477339975a4fab210edac3bc4c"}
Jul 13 21:48:17.454439 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:17.454409    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:17.543132 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:17.543102    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca\") pod \"2337d12a-2317-4847-bbe4-164ed1851846\" (UID: \"2337d12a-2317-4847-bbe4-164ed1851846\") "
Jul 13 21:48:17.543569 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:17.543539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2337d12a-2317-4847-bbe4-164ed1851846" (UID: "2337d12a-2317-4847-bbe4-164ed1851846"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:48:17.644639 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:17.644607    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2337d12a-2317-4847-bbe4-164ed1851846-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:48:18.330697 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:18.330666    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc"
Jul 13 21:48:18.330891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:18.330667    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc" event={"ID":"2337d12a-2317-4847-bbe4-164ed1851846","Type":"ContainerDied","Data":"824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20"}
Jul 13 21:48:18.330891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:18.330813    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="824cf465b228746cde99685d1f57d91a6a5ebe76d6271725e729b3993baf3d20"
Jul 13 21:48:24.981076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:24.981043    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:29.607886 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:29.607843    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:35.455828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:35.455736    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:35.557002 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:35.556964    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:40.342750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:40.342705    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:40.574340 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:40.574301    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:40.977698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:40.977664    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:41.281252 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:41.281170    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:41.386908 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:41.386878    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.081497 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.081464    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.189499 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.189470    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.281909 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.281880    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.458259 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.458229    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.783128 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.783047    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:44.884331 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:44.884290    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:48.598674 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:48.598633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:48.934451 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:48.934412    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:49.256889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:49.256796    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:49.462867 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:49.462835    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:52.135026 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:52.134993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:52.247894 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:52.247860    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:52.421465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:52.421434    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:52.523425 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:52.523396    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:52.895403 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:52.895369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:53.000518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:53.000485    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:48:56.628065 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:48:56.628018    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:05.616673 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:05.616633    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:05.719562 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:05.719532    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:10.649823 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.649786    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:49:10.650263 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.650113    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2337d12a-2317-4847-bbe4-164ed1851846" containerName="revoke-keys"
Jul 13 21:49:10.650263 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.650123    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2337d12a-2317-4847-bbe4-164ed1851846" containerName="revoke-keys"
Jul 13 21:49:10.650263 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.650182    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2337d12a-2317-4847-bbe4-164ed1851846" containerName="revoke-keys"
Jul 13 21:49:10.653114 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.653090    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.656562 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.656531    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"auth-test-model-8139bf-kserve-self-signed-certs\""
Jul 13 21:49:10.656562 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.656554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"default-dockercfg-nzmrl\""
Jul 13 21:49:10.656791 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.656593    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"kube-root-ca.crt\""
Jul 13 21:49:10.656791 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.656611    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"openshift-service-ca.crt\""
Jul 13 21:49:10.664189 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.664165    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:49:10.703580 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.703828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.703828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703671    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.703828 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4c4x\" (UniqueName: \"kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.704005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703831    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.704005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.703864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.728885 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.728857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:10.804710 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.804927 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804725    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.804927 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4c4x\" (UniqueName: \"kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.804927 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.804927 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.805159 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.804927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.805159 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.805130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.805325 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.805304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.805363 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.805318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.807170 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.807148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.807341 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.807324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.812422 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.812401    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4c4x\" (UniqueName: \"kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x\") pod \"auth-test-model-8139bf-kserve-9ff5456c-7n6sn\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") " pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:10.963231 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:10.963138    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:11.095979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:11.095950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:49:11.098416 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:49:11.098388    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b682503_7d42_45c0_9c78_6bd96c820f04.slice/crio-72a8396283e17627c28dd426b797ccd940fc1d4020f7398b6a03772a71c192e9 WatchSource:0}: Error finding container 72a8396283e17627c28dd426b797ccd940fc1d4020f7398b6a03772a71c192e9: Status 404 returned error can't find the container with id 72a8396283e17627c28dd426b797ccd940fc1d4020f7398b6a03772a71c192e9
Jul 13 21:49:11.565123 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:11.565083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" event={"ID":"3b682503-7d42-45c0-9c78-6bd96c820f04","Type":"ContainerStarted","Data":"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"}
Jul 13 21:49:11.565308 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:11.565134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" event={"ID":"3b682503-7d42-45c0-9c78-6bd96c820f04","Type":"ContainerStarted","Data":"72a8396283e17627c28dd426b797ccd940fc1d4020f7398b6a03772a71c192e9"}
Jul 13 21:49:11.582202 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:11.582149    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" podStartSLOduration=1.582133421 podStartE2EDuration="1.582133421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:49:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:49:11.58150703 +0000 UTC m=+2282.998433376" watchObservedRunningTime="2026-07-13 21:49:11.582133421 +0000 UTC m=+2282.999059765"
Jul 13 21:49:14.580040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:14.580008    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:14.680039 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:14.680004    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:49:20.963838 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:20.963792    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:20.964222 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:20.963864    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:20.975891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:20.975863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:21.619187 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:21.619157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:49:36.846605 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:49:36.846566    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:50:05.993827 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:50:05.993711    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:50:07.291808 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:50:07.291722    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:09.239779 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:09.239735    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:51:09.245644 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:09.245620    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:51:40.977164 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:40.977124    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:51:41.039008 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:41.038974    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:41.787957 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:41.787922    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:46.477405 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.477367    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:51:46.480592 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.480574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.483321 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.483300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"sub-test-model-8139bf-kserve-self-signed-certs\""
Jul 13 21:51:46.490332 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.490307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:51:46.542822 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.542785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.542822 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.542825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twcxs\" (UniqueName: \"kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.543109 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.542845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.543109 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.542916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.543109 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.542940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.543109 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.543016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.548528 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.548499    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:46.643648 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.643865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643689    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.643865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twcxs\" (UniqueName: \"kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.643865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.643865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643819    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.643865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.643850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.644138 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.644114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.644206 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.644159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.644271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.644253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.646055 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.646034    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.646212 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.646196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.651852 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.651820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twcxs\" (UniqueName: \"kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs\") pod \"sub-test-model-8139bf-kserve-89f444fb9-d6twk\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") " pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.790406 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.790313    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:46.921402 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:46.921374    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:51:46.923060 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:51:46.923027    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfeef906d_88a5_4629_adbd_64af243b6d8b.slice/crio-c444afb4c5133a67bf5217bda014d578ff1fb3a54dd9b9f92612065981a9b1e4 WatchSource:0}: Error finding container c444afb4c5133a67bf5217bda014d578ff1fb3a54dd9b9f92612065981a9b1e4: Status 404 returned error can't find the container with id c444afb4c5133a67bf5217bda014d578ff1fb3a54dd9b9f92612065981a9b1e4
Jul 13 21:51:47.259663 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:47.259624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" event={"ID":"feef906d-88a5-4629-adbd-64af243b6d8b","Type":"ContainerStarted","Data":"3fe217292dbd20e956abb7318e3732a3c095a51f76da3c52e6d5a00ce71a7e19"}
Jul 13 21:51:47.259883 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:47.259670    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" event={"ID":"feef906d-88a5-4629-adbd-64af243b6d8b","Type":"ContainerStarted","Data":"c444afb4c5133a67bf5217bda014d578ff1fb3a54dd9b9f92612065981a9b1e4"}
Jul 13 21:51:47.276698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:47.276645    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" podStartSLOduration=1.276631007 podStartE2EDuration="1.276631007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:51:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:51:47.276539042 +0000 UTC m=+2438.693465386" watchObservedRunningTime="2026-07-13 21:51:47.276631007 +0000 UTC m=+2438.693557351"
Jul 13 21:51:55.997179 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:55.997135    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" podUID="3b682503-7d42-45c0-9c78-6bd96c820f04" containerName="main" containerID="cri-o://4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec" gracePeriod=45
Jul 13 21:51:56.241904 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.241878    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:51:56.300994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.300898    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b682503-7d42-45c0-9c78-6bd96c820f04" containerID="4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec" exitCode=0
Jul 13 21:51:56.300994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.300948    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"
Jul 13 21:51:56.301220 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.300992    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" event={"ID":"3b682503-7d42-45c0-9c78-6bd96c820f04","Type":"ContainerDied","Data":"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"}
Jul 13 21:51:56.301220 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.301041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn" event={"ID":"3b682503-7d42-45c0-9c78-6bd96c820f04","Type":"ContainerDied","Data":"72a8396283e17627c28dd426b797ccd940fc1d4020f7398b6a03772a71c192e9"}
Jul 13 21:51:56.301220 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.301062    2577 scope.go:117] "RemoveContainer" containerID="4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"
Jul 13 21:51:56.309548 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.309529    2577 scope.go:117] "RemoveContainer" containerID="4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"
Jul 13 21:51:56.309830 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:51:56.309807    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec\": container with ID starting with 4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec not found: ID does not exist" containerID="4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"
Jul 13 21:51:56.309887 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.309843    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec"} err="failed to get container status \"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec\": rpc error: code = NotFound desc = could not find container \"4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec\": container with ID starting with 4020189b6e34bf0321ab417386cfd650514b4793016692876d2e0a72854fdbec not found: ID does not exist"
Jul 13 21:51:56.324575 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324625    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324876 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324695    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4c4x\" (UniqueName: \"kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324876 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324724    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs\") pod \"3b682503-7d42-45c0-9c78-6bd96c820f04\" (UID: \"3b682503-7d42-45c0-9c78-6bd96c820f04\") "
Jul 13 21:51:56.324978 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324902    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:56.324978 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324918    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home" (OuterVolumeSpecName: "home") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:56.324978 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.324949    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:56.326798 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.326737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm" (OuterVolumeSpecName: "dshm") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:56.326897 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.326842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x" (OuterVolumeSpecName: "kube-api-access-v4c4x") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "kube-api-access-v4c4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:51:56.326897 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.326849    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3b682503-7d42-45c0-9c78-6bd96c820f04" (UID: "3b682503-7d42-45c0-9c78-6bd96c820f04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:51:56.425698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425657    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b682503-7d42-45c0-9c78-6bd96c820f04-tls-certs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.425698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425689    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-model-cache\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.425698 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425701    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-home\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.425962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425712    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-dshm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.425962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425723    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3b682503-7d42-45c0-9c78-6bd96c820f04-tmp-dir\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.425962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.425733    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4c4x\" (UniqueName: \"kubernetes.io/projected/3b682503-7d42-45c0-9c78-6bd96c820f04-kube-api-access-v4c4x\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:51:56.622715 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.622679    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:51:56.628363 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.628337    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/auth-test-model-8139bf-kserve-9ff5456c-7n6sn"]
Jul 13 21:51:56.703997 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.703963    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:56.768374 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.768327    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:51:56.791432 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.791398    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:56.791432 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.791443    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:56.804505 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:56.804471    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:51:57.187618 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:57.187580    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b682503-7d42-45c0-9c78-6bd96c820f04" path="/var/lib/kubelet/pods/3b682503-7d42-45c0-9c78-6bd96c820f04/volumes"
Jul 13 21:51:57.319192 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:51:57.319151    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:52:00.852383 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:00.852330    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:52:07.437255 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:07.437214    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:52:47.451053 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:47.451018    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:52:47.473077 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:47.473045    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:52:48.492582 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:48.492553    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:52:57.260215 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.260182    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:52:57.260653 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.260479    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b682503-7d42-45c0-9c78-6bd96c820f04" containerName="main"
Jul 13 21:52:57.260653 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.260491    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b682503-7d42-45c0-9c78-6bd96c820f04" containerName="main"
Jul 13 21:52:57.260653 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.260555    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b682503-7d42-45c0-9c78-6bd96c820f04" containerName="main"
Jul 13 21:52:57.263470 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.263447    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.265948 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.265921    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-8139bf\"/\"rate-test-model-8139bf-kserve-self-signed-certs\""
Jul 13 21:52:57.272268 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.272243    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:52:57.282274 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.282225    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:52:57.352874 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.352845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqhnq\" (UniqueName: \"kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.353046 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.352890    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.353046 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.352939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.353046 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.352975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.353046 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.353016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.353192 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.353117    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454264 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqhnq\" (UniqueName: \"kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454327    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454538 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454793 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454740    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.454854 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.454832    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.456723 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.456694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.456945 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.456928    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.462078 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.462056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqhnq\" (UniqueName: \"kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq\") pod \"rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") " pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.573405 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.573308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:52:57.710186 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:57.710162    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:52:57.711988 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:52:57.711959    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf67588ed_4346_4b95_80a0_274b56640f16.slice/crio-576456fa8a75e3c735fc71a2391f4094e885e2df1bce4ee5cf824db845b69dd5 WatchSource:0}: Error finding container 576456fa8a75e3c735fc71a2391f4094e885e2df1bce4ee5cf824db845b69dd5: Status 404 returned error can't find the container with id 576456fa8a75e3c735fc71a2391f4094e885e2df1bce4ee5cf824db845b69dd5
Jul 13 21:52:58.594728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:58.594678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" event={"ID":"f67588ed-4346-4b95-80a0-274b56640f16","Type":"ContainerStarted","Data":"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"}
Jul 13 21:52:58.594728 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:58.594731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" event={"ID":"f67588ed-4346-4b95-80a0-274b56640f16","Type":"ContainerStarted","Data":"576456fa8a75e3c735fc71a2391f4094e885e2df1bce4ee5cf824db845b69dd5"}
Jul 13 21:52:58.610876 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:52:58.610822    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" podStartSLOduration=1.610806897 podStartE2EDuration="1.610806897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:52:58.609984048 +0000 UTC m=+2510.026910403" watchObservedRunningTime="2026-07-13 21:52:58.610806897 +0000 UTC m=+2510.027733263"
Jul 13 21:53:02.471042 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.470959    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" podUID="feef906d-88a5-4629-adbd-64af243b6d8b" containerName="main" containerID="cri-o://3fe217292dbd20e956abb7318e3732a3c095a51f76da3c52e6d5a00ce71a7e19" gracePeriod=45
Jul 13 21:53:02.615105 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.615075    2577 generic.go:356] "Generic (PLEG): container finished" podID="feef906d-88a5-4629-adbd-64af243b6d8b" containerID="3fe217292dbd20e956abb7318e3732a3c095a51f76da3c52e6d5a00ce71a7e19" exitCode=0
Jul 13 21:53:02.615278 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.615156    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" event={"ID":"feef906d-88a5-4629-adbd-64af243b6d8b","Type":"ContainerDied","Data":"3fe217292dbd20e956abb7318e3732a3c095a51f76da3c52e6d5a00ce71a7e19"}
Jul 13 21:53:02.732263 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.732194    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:53:02.904882 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.904847    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.904882 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.904886    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.905168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.904930    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twcxs\" (UniqueName: \"kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.905168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.904948    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.905168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.904987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.905168 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.905031    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs\") pod \"feef906d-88a5-4629-adbd-64af243b6d8b\" (UID: \"feef906d-88a5-4629-adbd-64af243b6d8b\") "
Jul 13 21:53:02.905380 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.905199    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home" (OuterVolumeSpecName: "home") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:02.905380 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.905250    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:02.905380 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.905271    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache" (OuterVolumeSpecName: "model-cache") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:02.907140 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.907113    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs" (OuterVolumeSpecName: "kube-api-access-twcxs") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "kube-api-access-twcxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:53:02.907243 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.907147    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:53:02.907243 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:02.907158    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm" (OuterVolumeSpecName: "dshm") pod "feef906d-88a5-4629-adbd-64af243b6d8b" (UID: "feef906d-88a5-4629-adbd-64af243b6d8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006485    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twcxs\" (UniqueName: \"kubernetes.io/projected/feef906d-88a5-4629-adbd-64af243b6d8b-kube-api-access-twcxs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006526    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-dshm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006540    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-model-cache\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006551    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/feef906d-88a5-4629-adbd-64af243b6d8b-tls-certs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006562    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-home\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.006596 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.006571    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/feef906d-88a5-4629-adbd-64af243b6d8b-tmp-dir\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.620943 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.620902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk" event={"ID":"feef906d-88a5-4629-adbd-64af243b6d8b","Type":"ContainerDied","Data":"c444afb4c5133a67bf5217bda014d578ff1fb3a54dd9b9f92612065981a9b1e4"}
Jul 13 21:53:03.621390 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.620959    2577 scope.go:117] "RemoveContainer" containerID="3fe217292dbd20e956abb7318e3732a3c095a51f76da3c52e6d5a00ce71a7e19"
Jul 13 21:53:03.621390 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.620914    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"
Jul 13 21:53:03.637656 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.637625    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:53:03.640640 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:03.640616    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/sub-test-model-8139bf-kserve-89f444fb9-d6twk"]
Jul 13 21:53:05.182831 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:05.182800    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="feef906d-88a5-4629-adbd-64af243b6d8b" path="/var/lib/kubelet/pods/feef906d-88a5-4629-adbd-64af243b6d8b/volumes"
Jul 13 21:53:07.574043 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:07.573989    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:53:07.574514 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:07.574131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:53:07.586754 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:07.586725    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:53:07.650659 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:07.650629    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:53:11.920215 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:11.920178    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:53:11.975789 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:11.975738    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:53:15.828962 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:15.828927    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:53:27.951935 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:53:27.951876    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:06.346539 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:06.346496    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:06.446517 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:06.446473    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:07.429052 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:07.429018    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:54:07.450404 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:07.450369    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:08.459646 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:08.459599    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:09.401430 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.401392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251036167344"]
Jul 13 21:54:09.401755 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.401739    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="feef906d-88a5-4629-adbd-64af243b6d8b" containerName="main"
Jul 13 21:54:09.401755 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.401770    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="feef906d-88a5-4629-adbd-64af243b6d8b" containerName="main"
Jul 13 21:54:09.401863 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.401853    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="feef906d-88a5-4629-adbd-64af243b6d8b" containerName="main"
Jul 13 21:54:09.404635 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.404615    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:09.407006 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.406980    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-sfqjm\""
Jul 13 21:54:09.410496 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.410464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251036167344"]
Jul 13 21:54:09.555245 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.555206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvh2k\" (UniqueName: \"kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k\") pod \"test-curl-7519-140251036167344\" (UID: \"cb5b8c34-d107-41f9-852d-6be986ded8d0\") " pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:09.656207 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.656110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvh2k\" (UniqueName: \"kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k\") pod \"test-curl-7519-140251036167344\" (UID: \"cb5b8c34-d107-41f9-852d-6be986ded8d0\") " pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:09.664184 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.664156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvh2k\" (UniqueName: \"kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k\") pod \"test-curl-7519-140251036167344\" (UID: \"cb5b8c34-d107-41f9-852d-6be986ded8d0\") " pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:09.714624 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.714587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:09.853378 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.853351    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251036167344"]
Jul 13 21:54:09.855445 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:09.855419    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb5b8c34_d107_41f9_852d_6be986ded8d0.slice/crio-3acd56cee402e1139abf7a8c735c838c13e4dd7579915d6a4dd3fafeeee28c5a WatchSource:0}: Error finding container 3acd56cee402e1139abf7a8c735c838c13e4dd7579915d6a4dd3fafeeee28c5a: Status 404 returned error can't find the container with id 3acd56cee402e1139abf7a8c735c838c13e4dd7579915d6a4dd3fafeeee28c5a
Jul 13 21:54:09.856619 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.856603    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:54:09.925885 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:09.925739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251036167344" event={"ID":"cb5b8c34-d107-41f9-852d-6be986ded8d0","Type":"ContainerStarted","Data":"3acd56cee402e1139abf7a8c735c838c13e4dd7579915d6a4dd3fafeeee28c5a"}
Jul 13 21:54:10.931884 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:10.931845    2577 generic.go:356] "Generic (PLEG): container finished" podID="cb5b8c34-d107-41f9-852d-6be986ded8d0" containerID="7e9059e431ae0c4e79f5ffdc9d4dd8413dd50c6055bc6ae993af94974c9cac4c" exitCode=0
Jul 13 21:54:10.932358 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:10.931922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251036167344" event={"ID":"cb5b8c34-d107-41f9-852d-6be986ded8d0","Type":"ContainerDied","Data":"7e9059e431ae0c4e79f5ffdc9d4dd8413dd50c6055bc6ae993af94974c9cac4c"}
Jul 13 21:54:12.079973 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.079943    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:12.097814 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.097787    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251036167344_cb5b8c34-d107-41f9-852d-6be986ded8d0/test-curl-7519-140251036167344/0.log"
Jul 13 21:54:12.119748 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.119704    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251036167344"]
Jul 13 21:54:12.123135 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.123110    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251036167344"]
Jul 13 21:54:12.177522 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.177483    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvh2k\" (UniqueName: \"kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k\") pod \"cb5b8c34-d107-41f9-852d-6be986ded8d0\" (UID: \"cb5b8c34-d107-41f9-852d-6be986ded8d0\") "
Jul 13 21:54:12.179651 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.179620    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k" (OuterVolumeSpecName: "kube-api-access-mvh2k") pod "cb5b8c34-d107-41f9-852d-6be986ded8d0" (UID: "cb5b8c34-d107-41f9-852d-6be986ded8d0"). InnerVolumeSpecName "kube-api-access-mvh2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:12.256091 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.256005    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251036173616"]
Jul 13 21:54:12.256316 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.256302    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb5b8c34-d107-41f9-852d-6be986ded8d0" containerName="test-curl-7519-140251036167344"
Jul 13 21:54:12.256398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.256317    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb5b8c34-d107-41f9-852d-6be986ded8d0" containerName="test-curl-7519-140251036167344"
Jul 13 21:54:12.256398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.256375    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb5b8c34-d107-41f9-852d-6be986ded8d0" containerName="test-curl-7519-140251036167344"
Jul 13 21:54:12.259212 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.259196    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:12.266675 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.266653    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251036173616"]
Jul 13 21:54:12.279140 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.279100    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvh2k\" (UniqueName: \"kubernetes.io/projected/cb5b8c34-d107-41f9-852d-6be986ded8d0-kube-api-access-mvh2k\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:12.380163 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.380120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mhf7\" (UniqueName: \"kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7\") pod \"test-curl-7519-140251036173616\" (UID: \"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4\") " pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:12.480566 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.480516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mhf7\" (UniqueName: \"kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7\") pod \"test-curl-7519-140251036173616\" (UID: \"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4\") " pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:12.488835 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.488809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mhf7\" (UniqueName: \"kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7\") pod \"test-curl-7519-140251036173616\" (UID: \"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4\") " pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:12.568431 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.568345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:12.697167 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.697130    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251036173616"]
Jul 13 21:54:12.700169 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:12.700145    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ba68c9f_54c4_4f3e_a9b4_af6e1dc8dbe4.slice/crio-530acca632c9101952578f86ebc8ee7067aba76540c6678129b65c2b1c6295cc WatchSource:0}: Error finding container 530acca632c9101952578f86ebc8ee7067aba76540c6678129b65c2b1c6295cc: Status 404 returned error can't find the container with id 530acca632c9101952578f86ebc8ee7067aba76540c6678129b65c2b1c6295cc
Jul 13 21:54:12.943290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.943253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251036173616" event={"ID":"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4","Type":"ContainerStarted","Data":"530acca632c9101952578f86ebc8ee7067aba76540c6678129b65c2b1c6295cc"}
Jul 13 21:54:12.944498 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.944474    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:12.944646 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:12.944476    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3acd56cee402e1139abf7a8c735c838c13e4dd7579915d6a4dd3fafeeee28c5a"
Jul 13 21:54:12.955177 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:12.955147    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251036167344\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="cb5b8c34-d107-41f9-852d-6be986ded8d0" pod="openshift-ingress/test-curl-7519-140251036167344"
Jul 13 21:54:13.182525 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:13.182485    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb5b8c34-d107-41f9-852d-6be986ded8d0" path="/var/lib/kubelet/pods/cb5b8c34-d107-41f9-852d-6be986ded8d0/volumes"
Jul 13 21:54:13.950734 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:13.950698    2577 generic.go:356] "Generic (PLEG): container finished" podID="9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" containerID="e793f877ac968083c2ffd6cc9b4a63fa91f683467c2151bfb072806f692e0bb6" exitCode=0
Jul 13 21:54:13.950931 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:13.950795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251036173616" event={"ID":"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4","Type":"ContainerDied","Data":"e793f877ac968083c2ffd6cc9b4a63fa91f683467c2151bfb072806f692e0bb6"}
Jul 13 21:54:15.097749 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.097721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:15.115857 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.115807    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251036173616_9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4/test-curl-7519-140251036173616/0.log"
Jul 13 21:54:15.139784 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.139728    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251036173616"]
Jul 13 21:54:15.142468 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.142443    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251036173616"]
Jul 13 21:54:15.208081 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.207990    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mhf7\" (UniqueName: \"kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7\") pod \"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4\" (UID: \"9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4\") "
Jul 13 21:54:15.210099 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.210074    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7" (OuterVolumeSpecName: "kube-api-access-2mhf7") pod "9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" (UID: "9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4"). InnerVolumeSpecName "kube-api-access-2mhf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:15.309026 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.308992    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:15.309295 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.309282    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" containerName="test-curl-7519-140251036173616"
Jul 13 21:54:15.309342 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.309296    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" containerName="test-curl-7519-140251036173616"
Jul 13 21:54:15.309380 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.309369    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" containerName="test-curl-7519-140251036173616"
Jul 13 21:54:15.309417 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.309403    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mhf7\" (UniqueName: \"kubernetes.io/projected/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4-kube-api-access-2mhf7\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:15.312172 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.312154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:15.317166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.317139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:15.410040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.410005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75btm\" (UniqueName: \"kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm\") pod \"test-curl-7519-140251035974224\" (UID: \"1bef281c-b199-405f-b5bb-39f5fa928250\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:15.483669 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.483589    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:15.511177 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.511136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75btm\" (UniqueName: \"kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm\") pod \"test-curl-7519-140251035974224\" (UID: \"1bef281c-b199-405f-b5bb-39f5fa928250\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:15.518998 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.518970    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75btm\" (UniqueName: \"kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm\") pod \"test-curl-7519-140251035974224\" (UID: \"1bef281c-b199-405f-b5bb-39f5fa928250\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:15.584222 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.584179    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:15.621865 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.621826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:15.755424 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.755399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:15.757797 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:15.757741    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bef281c_b199_405f_b5bb_39f5fa928250.slice/crio-1b97fe3dfb0007b2b4e61940a74d0cfecd9b75da55f40e80f40d400349d66a89 WatchSource:0}: Error finding container 1b97fe3dfb0007b2b4e61940a74d0cfecd9b75da55f40e80f40d400349d66a89: Status 404 returned error can't find the container with id 1b97fe3dfb0007b2b4e61940a74d0cfecd9b75da55f40e80f40d400349d66a89
Jul 13 21:54:15.961113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.961083    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251036173616"
Jul 13 21:54:15.961113 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.961111    2577 scope.go:117] "RemoveContainer" containerID="e793f877ac968083c2ffd6cc9b4a63fa91f683467c2151bfb072806f692e0bb6"
Jul 13 21:54:15.962505 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:15.962480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035974224" event={"ID":"1bef281c-b199-405f-b5bb-39f5fa928250","Type":"ContainerStarted","Data":"1b97fe3dfb0007b2b4e61940a74d0cfecd9b75da55f40e80f40d400349d66a89"}
Jul 13 21:54:16.971144 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:16.971106    2577 generic.go:356] "Generic (PLEG): container finished" podID="1bef281c-b199-405f-b5bb-39f5fa928250" containerID="532c1c73cbacca595f950e1bcb4bdd1e8ffa854d46d81a7516506615bd098a17" exitCode=0
Jul 13 21:54:16.971569 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:16.971193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035974224" event={"ID":"1bef281c-b199-405f-b5bb-39f5fa928250","Type":"ContainerDied","Data":"532c1c73cbacca595f950e1bcb4bdd1e8ffa854d46d81a7516506615bd098a17"}
Jul 13 21:54:17.182850 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:17.182819    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4" path="/var/lib/kubelet/pods/9ba68c9f-54c4-4f3e-a9b4-af6e1dc8dbe4/volumes"
Jul 13 21:54:18.109087 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.109056    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.128607 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.128581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251035974224_1bef281c-b199-405f-b5bb-39f5fa928250/test-curl-7519-140251035974224/0.log"
Jul 13 21:54:18.131888 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.131859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-75btm\" (UniqueName: \"kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm\") pod \"1bef281c-b199-405f-b5bb-39f5fa928250\" (UID: \"1bef281c-b199-405f-b5bb-39f5fa928250\") "
Jul 13 21:54:18.133970 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.133941    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm" (OuterVolumeSpecName: "kube-api-access-75btm") pod "1bef281c-b199-405f-b5bb-39f5fa928250" (UID: "1bef281c-b199-405f-b5bb-39f5fa928250"). InnerVolumeSpecName "kube-api-access-75btm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:18.152979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.152941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:18.157896 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.157857    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:18.232620 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.232531    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75btm\" (UniqueName: \"kubernetes.io/projected/1bef281c-b199-405f-b5bb-39f5fa928250-kube-api-access-75btm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:18.277045 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.277006    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:18.277410 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.277393    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bef281c-b199-405f-b5bb-39f5fa928250" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:18.277485 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.277412    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bef281c-b199-405f-b5bb-39f5fa928250" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:18.277539 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.277496    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bef281c-b199-405f-b5bb-39f5fa928250" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:18.280271 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.280245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.285941 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.285911    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:18.333338 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.333300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvfxp\" (UniqueName: \"kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp\") pod \"test-curl-7519-140251035974224\" (UID: \"4f827369-5fd3-4e50-af45-be491bbb628b\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.434517 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.434476    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvfxp\" (UniqueName: \"kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp\") pod \"test-curl-7519-140251035974224\" (UID: \"4f827369-5fd3-4e50-af45-be491bbb628b\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.442159 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.442125    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvfxp\" (UniqueName: \"kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp\") pod \"test-curl-7519-140251035974224\" (UID: \"4f827369-5fd3-4e50-af45-be491bbb628b\") " pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.590288 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.590198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.718746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.718659    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:18.720907 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:18.720876    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f827369_5fd3_4e50_af45_be491bbb628b.slice/crio-f432963cce282aea0b83d883adf8eac1e59f0fc8313fba22b999abf1669f97c3 WatchSource:0}: Error finding container f432963cce282aea0b83d883adf8eac1e59f0fc8313fba22b999abf1669f97c3: Status 404 returned error can't find the container with id f432963cce282aea0b83d883adf8eac1e59f0fc8313fba22b999abf1669f97c3
Jul 13 21:54:18.982494 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.982462    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:18.982494 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.982492    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b97fe3dfb0007b2b4e61940a74d0cfecd9b75da55f40e80f40d400349d66a89"
Jul 13 21:54:18.983649 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.983626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035974224" event={"ID":"4f827369-5fd3-4e50-af45-be491bbb628b","Type":"ContainerStarted","Data":"f432963cce282aea0b83d883adf8eac1e59f0fc8313fba22b999abf1669f97c3"}
Jul 13 21:54:18.993928 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:18.993893    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7519-140251035974224" oldPodUID="1bef281c-b199-405f-b5bb-39f5fa928250" podUID="4f827369-5fd3-4e50-af45-be491bbb628b"
Jul 13 21:54:19.182571 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:19.182539    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bef281c-b199-405f-b5bb-39f5fa928250" path="/var/lib/kubelet/pods/1bef281c-b199-405f-b5bb-39f5fa928250/volumes"
Jul 13 21:54:19.990503 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:19.990468    2577 generic.go:356] "Generic (PLEG): container finished" podID="4f827369-5fd3-4e50-af45-be491bbb628b" containerID="28f9a5d1655169daf5fd1d9337079c5a3305a60c7eab7c464529a3093ab2fa29" exitCode=0
Jul 13 21:54:19.990677 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:19.990539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035974224" event={"ID":"4f827369-5fd3-4e50-af45-be491bbb628b","Type":"ContainerDied","Data":"28f9a5d1655169daf5fd1d9337079c5a3305a60c7eab7c464529a3093ab2fa29"}
Jul 13 21:54:21.132368 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.132338    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:21.151959 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.151934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251035974224_4f827369-5fd3-4e50-af45-be491bbb628b/test-curl-7519-140251035974224/0.log"
Jul 13 21:54:21.155166 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.155142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvfxp\" (UniqueName: \"kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp\") pod \"4f827369-5fd3-4e50-af45-be491bbb628b\" (UID: \"4f827369-5fd3-4e50-af45-be491bbb628b\") "
Jul 13 21:54:21.157214 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.157179    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp" (OuterVolumeSpecName: "kube-api-access-nvfxp") pod "4f827369-5fd3-4e50-af45-be491bbb628b" (UID: "4f827369-5fd3-4e50-af45-be491bbb628b"). InnerVolumeSpecName "kube-api-access-nvfxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:21.175593 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.175565    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:21.183138 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.183114    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251035974224"]
Jul 13 21:54:21.256016 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.255929    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvfxp\" (UniqueName: \"kubernetes.io/projected/4f827369-5fd3-4e50-af45-be491bbb628b-kube-api-access-nvfxp\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:21.314517 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.314482    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:21.314837 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.314821    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:21.314889 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.314841    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:21.314925 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.314909    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" containerName="test-curl-7519-140251035974224"
Jul 13 21:54:21.317793 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.317775    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:21.325009 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.324852    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:21.357117 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.357081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc57p\" (UniqueName: \"kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p\") pod \"test-curl-7519-140251035975232\" (UID: \"d36d7144-1097-4300-afbf-ac6a1a522d24\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:21.457774 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.457734    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc57p\" (UniqueName: \"kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p\") pod \"test-curl-7519-140251035975232\" (UID: \"d36d7144-1097-4300-afbf-ac6a1a522d24\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:21.467949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.467919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc57p\" (UniqueName: \"kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p\") pod \"test-curl-7519-140251035975232\" (UID: \"d36d7144-1097-4300-afbf-ac6a1a522d24\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:21.627746 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.627651    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:21.754718 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:21.754692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:21.756586 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:21.756561    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd36d7144_1097_4300_afbf_ac6a1a522d24.slice/crio-cb29d74bf384c1bf1006bf4874c150882a936d3cd4410243e68bc54256ec6a9c WatchSource:0}: Error finding container cb29d74bf384c1bf1006bf4874c150882a936d3cd4410243e68bc54256ec6a9c: Status 404 returned error can't find the container with id cb29d74bf384c1bf1006bf4874c150882a936d3cd4410243e68bc54256ec6a9c
Jul 13 21:54:22.001199 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.001170    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:22.001398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.001168    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f432963cce282aea0b83d883adf8eac1e59f0fc8313fba22b999abf1669f97c3"
Jul 13 21:54:22.002490 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.002459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035975232" event={"ID":"d36d7144-1097-4300-afbf-ac6a1a522d24","Type":"ContainerStarted","Data":"cb29d74bf384c1bf1006bf4874c150882a936d3cd4410243e68bc54256ec6a9c"}
Jul 13 21:54:22.005989 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:22.005963    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035974224\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:22.449026 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.448991    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" podUID="f67588ed-4346-4b95-80a0-274b56640f16" containerName="main" containerID="cri-o://1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb" gracePeriod=45
Jul 13 21:54:22.699741 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.699676    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:54:22.701811 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:22.701787    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035974224\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:22.767561 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.767750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqhnq\" (UniqueName: \"kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.767750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.767750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767661    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.767750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767695    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.767750 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767733    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir\") pod \"f67588ed-4346-4b95-80a0-274b56640f16\" (UID: \"f67588ed-4346-4b95-80a0-274b56640f16\") "
Jul 13 21:54:22.768050 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home" (OuterVolumeSpecName: "home") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:22.768050 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.767958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache" (OuterVolumeSpecName: "model-cache") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:22.768050 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.768040    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-model-cache\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:22.768191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.768059    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-home\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:22.768191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.768068    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:22.769820 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.769796    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm" (OuterVolumeSpecName: "dshm") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:22.769908 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.769858    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:54:22.769908 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.769881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq" (OuterVolumeSpecName: "kube-api-access-qqhnq") pod "f67588ed-4346-4b95-80a0-274b56640f16" (UID: "f67588ed-4346-4b95-80a0-274b56640f16"). InnerVolumeSpecName "kube-api-access-qqhnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:22.868946 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.868904    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-dshm\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:22.868946 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.868942    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f67588ed-4346-4b95-80a0-274b56640f16-tls-certs\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:22.868946 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.868955    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f67588ed-4346-4b95-80a0-274b56640f16-tmp-dir\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:22.868946 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:22.868963    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqhnq\" (UniqueName: \"kubernetes.io/projected/f67588ed-4346-4b95-80a0-274b56640f16-kube-api-access-qqhnq\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.008497 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.008398    2577 generic.go:356] "Generic (PLEG): container finished" podID="f67588ed-4346-4b95-80a0-274b56640f16" containerID="1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb" exitCode=0
Jul 13 21:54:23.008497 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.008453    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"
Jul 13 21:54:23.008731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.008487    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" event={"ID":"f67588ed-4346-4b95-80a0-274b56640f16","Type":"ContainerDied","Data":"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"}
Jul 13 21:54:23.008731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.008538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2" event={"ID":"f67588ed-4346-4b95-80a0-274b56640f16","Type":"ContainerDied","Data":"576456fa8a75e3c735fc71a2391f4094e885e2df1bce4ee5cf824db845b69dd5"}
Jul 13 21:54:23.008731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.008565    2577 scope.go:117] "RemoveContainer" containerID="1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"
Jul 13 21:54:23.010001 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.009978    2577 generic.go:356] "Generic (PLEG): container finished" podID="d36d7144-1097-4300-afbf-ac6a1a522d24" containerID="0a2b8c413aaa49f4f7dcdfec42e142f99b9e9c16518a852ea68c2ab51444fcbd" exitCode=0
Jul 13 21:54:23.010092 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.010035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035975232" event={"ID":"d36d7144-1097-4300-afbf-ac6a1a522d24","Type":"ContainerDied","Data":"0a2b8c413aaa49f4f7dcdfec42e142f99b9e9c16518a852ea68c2ab51444fcbd"}
Jul 13 21:54:23.013623 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:23.013593    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035974224\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:23.017394 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.017379    2577 scope.go:117] "RemoveContainer" containerID="1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"
Jul 13 21:54:23.017637 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:23.017618    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb\": container with ID starting with 1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb not found: ID does not exist" containerID="1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"
Jul 13 21:54:23.017691 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.017651    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb"} err="failed to get container status \"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb\": rpc error: code = NotFound desc = could not find container \"1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb\": container with ID starting with 1e33ca225e4c843257b538bba03c3e650237f2acbb300886ab54221e083a84eb not found: ID does not exist"
Jul 13 21:54:23.027803 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:23.027780    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035974224\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:23.029435 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:23.029413    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035974224\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" pod="openshift-ingress/test-curl-7519-140251035974224"
Jul 13 21:54:23.040910 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.040886    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:54:23.044238 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.044214    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-8139bf/rate-test-model-8139bf-kserve-78c4bbd4bc-v54t2"]
Jul 13 21:54:23.182716 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.182685    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f827369-5fd3-4e50-af45-be491bbb628b" path="/var/lib/kubelet/pods/4f827369-5fd3-4e50-af45-be491bbb628b/volumes"
Jul 13 21:54:23.183101 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:23.183087    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f67588ed-4346-4b95-80a0-274b56640f16" path="/var/lib/kubelet/pods/f67588ed-4346-4b95-80a0-274b56640f16/volumes"
Jul 13 21:54:24.146369 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.146340    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.166804 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.166782    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251035975232_d36d7144-1097-4300-afbf-ac6a1a522d24/test-curl-7519-140251035975232/0.log"
Jul 13 21:54:24.179096 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.179076    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc57p\" (UniqueName: \"kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p\") pod \"d36d7144-1097-4300-afbf-ac6a1a522d24\" (UID: \"d36d7144-1097-4300-afbf-ac6a1a522d24\") "
Jul 13 21:54:24.181076 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.181044    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p" (OuterVolumeSpecName: "kube-api-access-pc57p") pod "d36d7144-1097-4300-afbf-ac6a1a522d24" (UID: "d36d7144-1097-4300-afbf-ac6a1a522d24"). InnerVolumeSpecName "kube-api-access-pc57p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:24.192209 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.192187    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:24.200290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.200266    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:24.280081 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.279998    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc57p\" (UniqueName: \"kubernetes.io/projected/d36d7144-1097-4300-afbf-ac6a1a522d24-kube-api-access-pc57p\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:24.318290 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318259    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:24.318546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318536    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f67588ed-4346-4b95-80a0-274b56640f16" containerName="main"
Jul 13 21:54:24.318588 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318548    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f67588ed-4346-4b95-80a0-274b56640f16" containerName="main"
Jul 13 21:54:24.318588 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318571    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36d7144-1097-4300-afbf-ac6a1a522d24" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:24.318588 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318578    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36d7144-1097-4300-afbf-ac6a1a522d24" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:24.318672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318625    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f67588ed-4346-4b95-80a0-274b56640f16" containerName="main"
Jul 13 21:54:24.318672 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.318632    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36d7144-1097-4300-afbf-ac6a1a522d24" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:24.321441 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.321425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.327513 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.327480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:24.380359 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.380328    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twbwk\" (UniqueName: \"kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk\") pod \"test-curl-7519-140251035975232\" (UID: \"f795e136-7414-4774-a2fc-4b27329d76b0\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.480817 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.480784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twbwk\" (UniqueName: \"kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk\") pod \"test-curl-7519-140251035975232\" (UID: \"f795e136-7414-4774-a2fc-4b27329d76b0\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.488502 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.488479    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twbwk\" (UniqueName: \"kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk\") pod \"test-curl-7519-140251035975232\" (UID: \"f795e136-7414-4774-a2fc-4b27329d76b0\") " pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.632082 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.631994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:24.761283 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:24.761255    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:24.763787 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:24.763746    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf795e136_7414_4774_a2fc_4b27329d76b0.slice/crio-9eb7aa4396cd0a07137c42867ed25a686b16bad5a5ac274ab3ffc97acec8d56f WatchSource:0}: Error finding container 9eb7aa4396cd0a07137c42867ed25a686b16bad5a5ac274ab3ffc97acec8d56f: Status 404 returned error can't find the container with id 9eb7aa4396cd0a07137c42867ed25a686b16bad5a5ac274ab3ffc97acec8d56f
Jul 13 21:54:25.021616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:25.021588    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:25.021813 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:25.021634    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb29d74bf384c1bf1006bf4874c150882a936d3cd4410243e68bc54256ec6a9c"
Jul 13 21:54:25.022940 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:25.022903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035975232" event={"ID":"f795e136-7414-4774-a2fc-4b27329d76b0","Type":"ContainerStarted","Data":"9eb7aa4396cd0a07137c42867ed25a686b16bad5a5ac274ab3ffc97acec8d56f"}
Jul 13 21:54:25.032822 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:25.032796    2577 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7519-140251035975232" oldPodUID="d36d7144-1097-4300-afbf-ac6a1a522d24" podUID="f795e136-7414-4774-a2fc-4b27329d76b0"
Jul 13 21:54:25.183257 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:25.183218    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d36d7144-1097-4300-afbf-ac6a1a522d24" path="/var/lib/kubelet/pods/d36d7144-1097-4300-afbf-ac6a1a522d24/volumes"
Jul 13 21:54:26.028849 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:26.028814    2577 generic.go:356] "Generic (PLEG): container finished" podID="f795e136-7414-4774-a2fc-4b27329d76b0" containerID="641e6ccd053ec5033d8b9fb691380c3cc7125892cbadd46cbe2a174dc6ced184" exitCode=0
Jul 13 21:54:26.029040 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:26.028906    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7519-140251035975232" event={"ID":"f795e136-7414-4774-a2fc-4b27329d76b0","Type":"ContainerDied","Data":"641e6ccd053ec5033d8b9fb691380c3cc7125892cbadd46cbe2a174dc6ced184"}
Jul 13 21:54:27.169072 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.169046    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:27.190806 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.190751    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7519-140251035975232_f795e136-7414-4774-a2fc-4b27329d76b0/test-curl-7519-140251035975232/0.log"
Jul 13 21:54:27.200963 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.200939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twbwk\" (UniqueName: \"kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk\") pod \"f795e136-7414-4774-a2fc-4b27329d76b0\" (UID: \"f795e136-7414-4774-a2fc-4b27329d76b0\") "
Jul 13 21:54:27.202994 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.202971    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk" (OuterVolumeSpecName: "kube-api-access-twbwk") pod "f795e136-7414-4774-a2fc-4b27329d76b0" (UID: "f795e136-7414-4774-a2fc-4b27329d76b0"). InnerVolumeSpecName "kube-api-access-twbwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:27.213118 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.213092    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:27.217173 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.217153    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7519-140251035975232"]
Jul 13 21:54:27.302535 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.302452    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twbwk\" (UniqueName: \"kubernetes.io/projected/f795e136-7414-4774-a2fc-4b27329d76b0-kube-api-access-twbwk\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:27.398716 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.398683    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"]
Jul 13 21:54:27.399011 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.398998    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:27.399073 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.399013    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:27.399073 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.399063    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" containerName="test-curl-7519-140251035975232"
Jul 13 21:54:27.402033 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.402015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:27.405280 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.405255    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:54:27.405437 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.405421    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:54:27.429394 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.429365    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"]
Jul 13 21:54:27.504279 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.504240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h\" (UID: \"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:27.605487 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.605387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h\" (UID: \"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:27.606095 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.606069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h\" (UID: \"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:27.712279 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.712246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:27.839711 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.839672    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"]
Jul 13 21:54:27.841455 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:27.841431    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a5f24c_1d3e_4700_adaf_6484a1fb7b9a.slice/crio-a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df WatchSource:0}: Error finding container a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df: Status 404 returned error can't find the container with id a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df
Jul 13 21:54:27.978891 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:27.978857    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:28.039755 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:28.039732    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:28.039949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:28.039797    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9eb7aa4396cd0a07137c42867ed25a686b16bad5a5ac274ab3ffc97acec8d56f"
Jul 13 21:54:28.041200 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:28.041036    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h" event={"ID":"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a","Type":"ContainerStarted","Data":"a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df"}
Jul 13 21:54:28.050856 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:28.050827    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035975232\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:28.079971 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:28.079939    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:29.047108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:29.047075    2577 generic.go:356] "Generic (PLEG): container finished" podID="09a5f24c-1d3e-4700-adaf-6484a1fb7b9a" containerID="c2c2a8ebf226c17c1720ee9ba9f2138b19a04069441e7acc6b44e94f5068af42" exitCode=0
Jul 13 21:54:29.047496 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:29.047134    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h" event={"ID":"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a","Type":"ContainerDied","Data":"c2c2a8ebf226c17c1720ee9ba9f2138b19a04069441e7acc6b44e94f5068af42"}
Jul 13 21:54:29.049344 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:29.049319    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035975232\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:29.181164 ip-10-0-143-242 kubenswrapper[2577]: E0713 21:54:29.181119    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7519-140251035975232\" is forbidden: User \"system:node:ip-10-0-143-242.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-242.ec2.internal' and this object" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" pod="openshift-ingress/test-curl-7519-140251035975232"
Jul 13 21:54:29.182196 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:29.182174    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f795e136-7414-4774-a2fc-4b27329d76b0" path="/var/lib/kubelet/pods/f795e136-7414-4774-a2fc-4b27329d76b0/volumes"
Jul 13 21:54:30.188296 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:30.188268    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:30.229198 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:30.229163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca\") pod \"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a\" (UID: \"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a\") "
Jul 13 21:54:30.229534 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:30.229507    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "09a5f24c-1d3e-4700-adaf-6484a1fb7b9a" (UID: "09a5f24c-1d3e-4700-adaf-6484a1fb7b9a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:54:30.330199 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:30.330103    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5f24c-1d3e-4700-adaf-6484a1fb7b9a-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:31.063204 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:31.063172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h"
Jul 13 21:54:31.063356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:31.063174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h" event={"ID":"09a5f24c-1d3e-4700-adaf-6484a1fb7b9a","Type":"ContainerDied","Data":"a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df"}
Jul 13 21:54:31.063356 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:31.063322    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2aa06ec0f1928160ec05cef3da88456cc0a066e62ded8224fec84129218c8df"
Jul 13 21:54:37.883009 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:37.882979    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:42.497794 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:42.497747    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:48.260005 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:48.259968    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:48.360916 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:48.360885    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:53.270398 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.270367    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:53.497821 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.497789    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:53.687408 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.687368    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"]
Jul 13 21:54:53.687805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.687788    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a5f24c-1d3e-4700-adaf-6484a1fb7b9a" containerName="revoke-keys"
Jul 13 21:54:53.687805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.687806    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a5f24c-1d3e-4700-adaf-6484a1fb7b9a" containerName="revoke-keys"
Jul 13 21:54:53.687975 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.687870    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a5f24c-1d3e-4700-adaf-6484a1fb7b9a" containerName="revoke-keys"
Jul 13 21:54:53.690900 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.690880    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:53.693733 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.693717    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:54:53.694805 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.694780    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:54:53.704867 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.704840    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"]
Jul 13 21:54:53.822546 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.822509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2\" (UID: \"63ee25f9-7355-42f7-85c7-59f67833d725\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:53.923812 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.923745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2\" (UID: \"63ee25f9-7355-42f7-85c7-59f67833d725\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:53.924380 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:53.924359    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2\" (UID: \"63ee25f9-7355-42f7-85c7-59f67833d725\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:54.000126 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:54.000038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:54.129662 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:54.129637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"]
Jul 13 21:54:54.132010 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:54:54.131978    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ee25f9_7355_42f7_85c7_59f67833d725.slice/crio-216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427 WatchSource:0}: Error finding container 216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427: Status 404 returned error can't find the container with id 216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427
Jul 13 21:54:54.170003 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:54.169975    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2" event={"ID":"63ee25f9-7355-42f7-85c7-59f67833d725","Type":"ContainerStarted","Data":"216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427"}
Jul 13 21:54:54.203090 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:54.203051    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:54.303616 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:54.303530    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:54:55.176152 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:55.176063    2577 generic.go:356] "Generic (PLEG): container finished" podID="63ee25f9-7355-42f7-85c7-59f67833d725" containerID="ba937c0a22d12c397097acc005af387d28e4b450a41fac0c130ab84bdd5bf81e" exitCode=0
Jul 13 21:54:55.176297 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:55.176147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2" event={"ID":"63ee25f9-7355-42f7-85c7-59f67833d725","Type":"ContainerDied","Data":"ba937c0a22d12c397097acc005af387d28e4b450a41fac0c130ab84bdd5bf81e"}
Jul 13 21:54:56.310631 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:56.310605    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:56.445214 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:56.445119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca\") pod \"63ee25f9-7355-42f7-85c7-59f67833d725\" (UID: \"63ee25f9-7355-42f7-85c7-59f67833d725\") "
Jul 13 21:54:56.445492 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:56.445467    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63ee25f9-7355-42f7-85c7-59f67833d725" (UID: "63ee25f9-7355-42f7-85c7-59f67833d725"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:54:56.546312 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:56.546267    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ee25f9-7355-42f7-85c7-59f67833d725-maas-api-service-ca\") on node \"ip-10-0-143-242.ec2.internal\" DevicePath \"\""
Jul 13 21:54:57.185866 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:57.185831    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2" event={"ID":"63ee25f9-7355-42f7-85c7-59f67833d725","Type":"ContainerDied","Data":"216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427"}
Jul 13 21:54:57.185866 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:57.185867    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2"
Jul 13 21:54:57.186073 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:54:57.185881    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="216607af12335fbe6fc4da4291d38be79ffd56ef35cdc0142cc79beacfb79427"
Jul 13 21:55:04.196285 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:04.196245    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:06.725812 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:06.725755    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:06.823029 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:06.822993    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:08.744583 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:08.744549    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:14.793797 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:14.793753    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:14.864164 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:14.864128    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:19.456112 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:19.456079    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:19.704568 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:19.704535    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:20.171619 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:20.171551    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b5dd9dbdc-7csqh_31f13755-f435-459a-b1b6-c4680209ed7f/authorino/0.log"
Jul 13 21:55:25.337474 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:25.337426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5988d4cf74-28fvc_c7a28d92-74ca-4712-a0ad-52307374782c/manager/0.log"
Jul 13 21:55:25.695093 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:25.695070    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-27m6v_8eb07a66-8351-4e18-8dbe-3c3eb8c188b0/copy-manifests/0.log"
Jul 13 21:55:25.708251 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:25.708202    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-27m6v_8eb07a66-8351-4e18-8dbe-3c3eb8c188b0/manager/0.log"
Jul 13 21:55:26.552729 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:26.552701    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732985-znj75_7c05ee3a-9be4-44d7-8cb6-027d2dcd7d8b/cleanup/0.log"
Jul 13 21:55:26.772952 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:26.772920    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-5278c205-t56tv_6f1bed64-be6d-47f6-828a-c634c52dee14/revoke-keys/0.log"
Jul 13 21:55:26.882556 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:26.882480    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-459a1d-jf84v_337f4fc1-837a-4c31-a505-fd9e662dab05/revoke-keys/0.log"
Jul 13 21:55:26.993563 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:26.993532    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-7b2acc-26tjc_2337d12a-2317-4847-bbe4-164ed1851846/revoke-keys/0.log"
Jul 13 21:55:27.105295 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:27.105268    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-f9b92c93-gkwpr_21538795-896a-4abb-ba61-200ad4f5c3c5/revoke-keys/0.log"
Jul 13 21:55:27.329112 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:27.329085    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-f491fd-ztvpp_668ae9e0-581e-4a25-b2c1-dde541b5a4cc/revoke-keys/0.log"
Jul 13 21:55:27.551190 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:27.551161    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-2b4e0142-bhqtl_c907c10a-7287-4cf6-a69e-ae3e03f5d9b0/revoke-keys/0.log"
Jul 13 21:55:27.797802 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:27.797745    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-8139bf-9pc6h_09a5f24c-1d3e-4700-adaf-6484a1fb7b9a/revoke-keys/0.log"
Jul 13 21:55:27.910306 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:27.910278    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-6c68d6-djdw2_63ee25f9-7355-42f7-85c7-59f67833d725/revoke-keys/0.log"
Jul 13 21:55:29.292139 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:29.292109    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b5dd9dbdc-7csqh_31f13755-f435-459a-b1b6-c4680209ed7f/authorino/0.log"
Jul 13 21:55:29.632154 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:29.632077    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dszdc_7d561b14-d7da-4ad5-b242-9941805cbafc/kuadrant-console-plugin/0.log"
Jul 13 21:55:30.708726 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:30.708699    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-9ntvp_fb5ef059-5257-4c47-a506-fea4867102bf/kube-auth-proxy/0.log"
Jul 13 21:55:31.087069 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:31.086994    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qzqz6_337a357f-6414-4e0e-be7b-b32abebd8b0c/payload-pre-processing/0.log"
Jul 13 21:55:31.680217 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:31.680184    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-6xdpm_8195d77e-7de4-46c2-be92-a7be07977460/main/0.log"
Jul 13 21:55:32.127365 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:32.127276    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5k9zp_b763bfbf-63d7-41b6-8222-5e0ae402ee82/main/0.log"
Jul 13 21:55:32.242549 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:32.242519    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-f6wkl_c0578bf0-a990-480a-ad6c-2baee5564d29/main/0.log"
Jul 13 21:55:32.759941 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:32.759913    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:32.815820 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:32.815780    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:36.064598 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:36.064567    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:36.133157 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:36.133120    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log" limit=1024
Jul 13 21:55:39.133083 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:39.133055    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bjjbg_7476560c-3a7d-42bc-88a6-7f2492d06a09/global-pull-secret-syncer/0.log"
Jul 13 21:55:39.258379 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:39.258336    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-26wqg_6f544454-66d5-4dc8-834b-f818696cdbf4/konnectivity-agent/0.log"
Jul 13 21:55:39.390724 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:39.390695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-242.ec2.internal_0ac4ccc85f99651906e4767386806ad9/haproxy/0.log"
Jul 13 21:55:44.347677 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:44.347638    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b5dd9dbdc-7csqh_31f13755-f435-459a-b1b6-c4680209ed7f/authorino/0.log"
Jul 13 21:55:44.448539 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:44.448510    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dszdc_7d561b14-d7da-4ad5-b242-9941805cbafc/kuadrant-console-plugin/0.log"
Jul 13 21:55:47.157983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:47.157955    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c5pcj_ada5e595-ce6b-4486-8a21-60e7a7ffd4c4/node-exporter/0.log"
Jul 13 21:55:47.184084 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:47.184058    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c5pcj_ada5e595-ce6b-4486-8a21-60e7a7ffd4c4/kube-rbac-proxy/0.log"
Jul 13 21:55:47.205883 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:47.205863    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c5pcj_ada5e595-ce6b-4486-8a21-60e7a7ffd4c4/init-textfile/0.log"
Jul 13 21:55:49.256108 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.256074    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"]
Jul 13 21:55:49.256529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.256437    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63ee25f9-7355-42f7-85c7-59f67833d725" containerName="revoke-keys"
Jul 13 21:55:49.256529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.256453    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="63ee25f9-7355-42f7-85c7-59f67833d725" containerName="revoke-keys"
Jul 13 21:55:49.256529 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.256514    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="63ee25f9-7355-42f7-85c7-59f67833d725" containerName="revoke-keys"
Jul 13 21:55:49.259348 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.259329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.261793 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.261752    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59z9w\"/\"kube-root-ca.crt\""
Jul 13 21:55:49.261919 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.261815    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59z9w\"/\"openshift-service-ca.crt\""
Jul 13 21:55:49.262624 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.262606    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-59z9w\"/\"default-dockercfg-lj8pz\""
Jul 13 21:55:49.268921 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.268889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"]
Jul 13 21:55:49.378435 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.378403    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-proc\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.378435 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.378442    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztq5g\" (UniqueName: \"kubernetes.io/projected/401a92f1-6d7f-47c3-a6d6-c4aee526e447-kube-api-access-ztq5g\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.378661 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.378470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-lib-modules\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.378661 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.378533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-podres\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.378661 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.378581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-sys\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.479908 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.479870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-lib-modules\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.479908 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.479910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-podres\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.479953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-sys\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-proc\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-lib-modules\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-proc\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-sys\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/401a92f1-6d7f-47c3-a6d6-c4aee526e447-podres\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.480162 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.480119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztq5g\" (UniqueName: \"kubernetes.io/projected/401a92f1-6d7f-47c3-a6d6-c4aee526e447-kube-api-access-ztq5g\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.487405 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.487379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztq5g\" (UniqueName: \"kubernetes.io/projected/401a92f1-6d7f-47c3-a6d6-c4aee526e447-kube-api-access-ztq5g\") pod \"perf-node-gather-daemonset-zxvwj\" (UID: \"401a92f1-6d7f-47c3-a6d6-c4aee526e447\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.569731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.569634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:49.696236 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:49.696213    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"]
Jul 13 21:55:49.698564 ip-10-0-143-242 kubenswrapper[2577]: W0713 21:55:49.698540    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod401a92f1_6d7f_47c3_a6d6_c4aee526e447.slice/crio-250df5cd4dad8b51292dca12df47c6f4336483f7ff6b67a94e4138cf219f2380 WatchSource:0}: Error finding container 250df5cd4dad8b51292dca12df47c6f4336483f7ff6b67a94e4138cf219f2380: Status 404 returned error can't find the container with id 250df5cd4dad8b51292dca12df47c6f4336483f7ff6b67a94e4138cf219f2380
Jul 13 21:55:50.442360 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:50.442322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj" event={"ID":"401a92f1-6d7f-47c3-a6d6-c4aee526e447","Type":"ContainerStarted","Data":"8fd2c2bf932a50566f3970c66648a9118da4eb630000e54abffb1e9eb66f224e"}
Jul 13 21:55:50.442801 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:50.442373    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj" event={"ID":"401a92f1-6d7f-47c3-a6d6-c4aee526e447","Type":"ContainerStarted","Data":"250df5cd4dad8b51292dca12df47c6f4336483f7ff6b67a94e4138cf219f2380"}
Jul 13 21:55:50.442801 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:50.442452    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:50.458543 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:50.458495    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj" podStartSLOduration=1.458478702 podStartE2EDuration="1.458478702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:55:50.457791806 +0000 UTC m=+2681.874718149" watchObservedRunningTime="2026-07-13 21:55:50.458478702 +0000 UTC m=+2681.875405045"
Jul 13 21:55:51.821967 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:51.821934    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kqxx5_7c927257-e779-4148-a577-c9bec9ad63f2/dns/0.log"
Jul 13 21:55:51.841267 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:51.841242    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kqxx5_7c927257-e779-4148-a577-c9bec9ad63f2/kube-rbac-proxy/0.log"
Jul 13 21:55:51.952740 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:51.952711    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zngbn_0729b1c1-8389-4d40-ab63-87e52c39942e/dns-node-resolver/0.log"
Jul 13 21:55:52.454581 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:52.454553    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-567dfb87dc-cj76z_adac3e94-350e-48e3-a19b-fa521ffb5884/registry/0.log"
Jul 13 21:55:52.473003 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:52.472978    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-98x85_23115f88-178a-4e5b-9c3f-34a160fb24e3/node-ca/0.log"
Jul 13 21:55:53.563409 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:53.563351    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-9ntvp_fb5ef059-5257-4c47-a506-fea4867102bf/kube-auth-proxy/0.log"
Jul 13 21:55:53.707704 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:53.707668    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qzqz6_337a357f-6414-4e0e-be7b-b32abebd8b0c/payload-pre-processing/0.log"
Jul 13 21:55:54.436191 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:54.436160    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8w765_edb8e8a1-8d39-4f54-9858-158909fbd6fb/serve-healthcheck-canary/0.log"
Jul 13 21:55:55.071035 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:55.071002    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qcwbl_c69d4585-bbb0-424c-a341-dc2ac1987cc4/kube-rbac-proxy/0.log"
Jul 13 21:55:55.092675 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:55.092651    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qcwbl_c69d4585-bbb0-424c-a341-dc2ac1987cc4/exporter/0.log"
Jul 13 21:55:55.111671 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:55.111648    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qcwbl_c69d4585-bbb0-424c-a341-dc2ac1987cc4/extractor/0.log"
Jul 13 21:55:56.456787 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:56.456750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-zxvwj"
Jul 13 21:55:57.477772 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:57.477729    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5988d4cf74-28fvc_c7a28d92-74ca-4712-a0ad-52307374782c/manager/0.log"
Jul 13 21:55:57.677997 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:57.677947    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-27m6v_8eb07a66-8351-4e18-8dbe-3c3eb8c188b0/manager/0.log"
Jul 13 21:55:57.693451 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:57.693426    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-27m6v_8eb07a66-8351-4e18-8dbe-3c3eb8c188b0/copy-manifests/0.log"
Jul 13 21:55:58.967466 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:58.967411    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-575f94fb69-vbbdc_81664d0b-8269-4fe5-b103-9e98259a8a29/manager/0.log"
Jul 13 21:55:59.900983 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:55:59.900957    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-9ntvp_fb5ef059-5257-4c47-a506-fea4867102bf/kube-auth-proxy/0.log"
Jul 13 21:56:00.041044 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:00.041009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qzqz6_337a357f-6414-4e0e-be7b-b32abebd8b0c/payload-pre-processing/0.log"
Jul 13 21:56:01.436731 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:01.436702    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-9ntvp_fb5ef059-5257-4c47-a506-fea4867102bf/kube-auth-proxy/0.log"
Jul 13 21:56:01.566917 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:01.566837    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-qzqz6_337a357f-6414-4e0e-be7b-b32abebd8b0c/payload-pre-processing/0.log"
Jul 13 21:56:04.222967 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:04.222924    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-2bm8d_0d554ace-4ba7-438f-a4e0-c14b23e81b70/migrator/0.log"
Jul 13 21:56:04.244481 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:04.244448    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-2bm8d_0d554ace-4ba7-438f-a4e0-c14b23e81b70/graceful-termination/0.log"
Jul 13 21:56:05.719557 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.719523    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7zgs6_49fe447d-9430-4e99-a587-f23eac1c8b03/kube-multus/0.log"
Jul 13 21:56:05.739949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.739923    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/kube-multus-additional-cni-plugins/0.log"
Jul 13 21:56:05.760182 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.760159    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/egress-router-binary-copy/0.log"
Jul 13 21:56:05.781261 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.781235    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/cni-plugins/0.log"
Jul 13 21:56:05.800648 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.800623    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/bond-cni-plugin/0.log"
Jul 13 21:56:05.820244 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.820220    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/routeoverride-cni/0.log"
Jul 13 21:56:05.839518 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.839494    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/whereabouts-cni-bincopy/0.log"
Jul 13 21:56:05.858180 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:05.858153    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9msh7_f79973c8-85c1-4284-b467-3788edeb087a/whereabouts-cni/0.log"
Jul 13 21:56:06.344950 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:06.344859    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzds7_fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab/network-metrics-daemon/0.log"
Jul 13 21:56:06.362465 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:06.362437    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rzds7_fe8a1ebb-d6f0-47a3-8de3-ce072eba0dab/kube-rbac-proxy/0.log"
Jul 13 21:56:07.513979 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.513952    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-controller/0.log"
Jul 13 21:56:07.532355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.532331    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/1.log"
Jul 13 21:56:07.532355 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.532358    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovn-acl-logging/0.log"
Jul 13 21:56:07.549115 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.549088    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/kube-rbac-proxy-node/0.log"
Jul 13 21:56:07.571985 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.571961    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/kube-rbac-proxy-ovn-metrics/0.log"
Jul 13 21:56:07.593943 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.593915    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/northd/0.log"
Jul 13 21:56:07.616633 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.616607    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/nbdb/0.log"
Jul 13 21:56:07.644949 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.644924    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/sbdb/0.log"
Jul 13 21:56:07.697613 ip-10-0-143-242 kubenswrapper[2577]: I0713 21:56:07.697585    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v6skj_92d6083b-543c-4039-8846-0c64da058fdc/ovnkube-controller/0.log"