Jul 14 21:50:57.675063 ip-10-0-130-163 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 21:50:58.122017 ip-10-0-130-163 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 21:50:58.122017 ip-10-0-130-163 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 21:50:58.122017 ip-10-0-130-163 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 21:50:58.122017 ip-10-0-130-163 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 21:50:58.122017 ip-10-0-130-163 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 21:50:58.123072 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.122977    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 21:50:58.128212 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128187    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.128212 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128210    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.128212 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128214    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.128212 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128218    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128221    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128224    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128226    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128229    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128232    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128234    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128237    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128240    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128244    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128246    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128249    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128252    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128255    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128258    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128261    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128269    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128272    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128274    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128277    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.128343 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128280    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128282    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128285    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128287    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128290    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128293    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128295    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128298    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128300    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128304    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128306    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128309    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128312    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128315    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128318    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128320    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128323    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128326    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128328    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128331    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.128824 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128334    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128337    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128339    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128342    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128344    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128347    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128350    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128353    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128356    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128358    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128361    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128363    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128366    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128369    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128372    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128374    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128377    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128380    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128385    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128388    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.129316 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128393    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128396    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128399    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128402    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128405    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128407    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128410    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128412    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128416    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128420    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128424    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128426    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128429    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128431    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128434    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128437    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128439    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128442    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128445    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.129810 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128448    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128451    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128454    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128458    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128461    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128464    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128467    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128470    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128473    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128475    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128478    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128480    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128483    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128485    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128489    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128491    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.128494    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129252    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129259    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.130276 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129262    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129265    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129268    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129271    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129274    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129277    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129279    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129282    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129285    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129287    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129290    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129293    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129296    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129298    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129301    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129305    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129309    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129312    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129316    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.130821 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129318    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129322    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129325    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129328    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129331    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129334    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129336    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129339    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129342    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129345    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129348    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129351    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129353    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129356    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129359    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129364    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129367    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129371    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129374    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129378    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.131294 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129381    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129383    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129386    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129388    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129391    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129393    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129396    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129399    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129401    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129404    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129406    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129409    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129411    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129414    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129416    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129419    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129422    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129425    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129427    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129430    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.131790 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129433    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129435    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129438    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129440    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129443    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129447    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129450    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129452    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129456    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129459    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129463    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129466    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129469    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129472    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129474    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129477    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129480    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129483    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129485    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.132278 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129488    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129490    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129493    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129496    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129498    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129501    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129504    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129506    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129509    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129511    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129514    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129517    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129519    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129522    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129524    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129527    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129529    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129532    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129534    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129972    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.132728 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129978    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129980    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129983    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129992    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129995    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.129998    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130001    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130003    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130006    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130014    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130016    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130019    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130022    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130024    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130027    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130029    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130032    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130036    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130040    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130043    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.133227 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130046    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130049    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130052    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130055    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130057    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130060    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130063    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130065    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130068    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130070    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130073    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130075    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130078    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130080    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130083    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130086    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130089    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130091    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130094    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130096    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.133702 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130099    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130103    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130105    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130108    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130111    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130113    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130116    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130118    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130121    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130123    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130127    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130130    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130132    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130135    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130138    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130141    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130143    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130146    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130149    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130151    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.134231 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130161    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130165    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130167    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130170    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130173    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130175    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130178    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130180    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130183    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130186    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130188    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130191    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130195    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130200    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130203    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130206    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130210    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130213    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130216    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.134718 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130219    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130222    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130226    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130228    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130231    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130233    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130235    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130238    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130241    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130243    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130246    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130248    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130251    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130253    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130256    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130259    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130261    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130263    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.130266    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130351    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130358    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 21:50:58.135202 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130366    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130384    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130401    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130405    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130409    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130415    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130418    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130421    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130425    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130428    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130431    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130434    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130437    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130443    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130446    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130449    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130456    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130459    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130462    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130465    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130469    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130473    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130476    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 21:50:58.135759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130479    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130482    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130485    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130488    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130491    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130494    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130497    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130501    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130504    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130507    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130510    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130513    2570 flags.go:64] FLAG: --enable-server="true"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130517    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130522    2570 flags.go:64] FLAG: --event-burst="100"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130527    2570 flags.go:64] FLAG: --event-qps="50"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130530    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130533    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130536    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130540    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130542    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130546    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130548    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130553    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130556    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130559    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 21:50:58.136320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130562    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130565    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130567    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130570    2570 flags.go:64] FLAG: --feature-gates=""
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130574    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130577    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130580    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130584    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130587    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130589    2570 flags.go:64] FLAG: --help="false"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130592    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130595    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130598    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130601    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130605    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130608    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130611    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130614    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130617    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130620    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130624    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130628    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130632    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130635    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 21:50:58.136957 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130638    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130641    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130654    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130658    2570 flags.go:64] FLAG: --lock-file=""
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130660    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130665    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130669    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130674    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130677    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130680    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130683    2570 flags.go:64] FLAG: --logging-format="text"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130686    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130689    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130692    2570 flags.go:64] FLAG: --manifest-url=""
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130695    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130700    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130703    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130707    2570 flags.go:64] FLAG: --max-pods="110"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130710    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130713    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130716    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130719    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130722    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130725    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130728    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 21:50:58.137557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130734    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130737    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130740    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130767    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130773    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130779    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130782    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130786    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130789    2570 flags.go:64] FLAG: --port="10250"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130792    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130795    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-066ff2a1b4800e2b4"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130798    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130801    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130805    2570 flags.go:64] FLAG: --register-node="true"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130809    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130813    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130816    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130819    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130822    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130826    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130829    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130832    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130835    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130838    2570 flags.go:64] FLAG: --runonce="false"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130841    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 21:50:58.138197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130844    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130848    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130850    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130853    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130856    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130859    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130862    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130865    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130868    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130870    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130874    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130878    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130881    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130884    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130890    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130892    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130895    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130899    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130902    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130905    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130910    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130913    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130916    2570 flags.go:64] FLAG: --v="2"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130920    2570 flags.go:64] FLAG: --version="false"
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130924    2570 flags.go:64] FLAG: --vmodule=""
Jul 14 21:50:58.138811 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130928    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.130931    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131049    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131053    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131056    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131059    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131062    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131065    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131067    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131070    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131073    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131075    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131078    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131080    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131083    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131086    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131088    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131091    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131094    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131096    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.139449 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131106    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131109    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131112    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131114    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131117    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131121    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131125    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131129    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131132    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131134    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131138    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131142    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131145    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131148    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131151    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131153    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131156    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131158    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131161    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.139957 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131163    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131166    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131169    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131171    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131174    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131176    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131179    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131181    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131184    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131186    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131189    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131191    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131194    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131196    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131205    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131207    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131210    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131213    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131216    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.140416 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131218    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131223    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131227    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131230    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131232    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131235    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131238    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131241    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131244    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131247    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131249    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131252    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131254    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131257    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131259    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131262    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131264    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131267    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131269    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131272    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.140889 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131274    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131277    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131279    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131282    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131284    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131287    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131289    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131292    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131305    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131308    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131311    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131313    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131316    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131320    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131322    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131325    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131327    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131330    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131333    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131336    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.141371 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131338    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131342    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.131345    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.131351    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.138966    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.138986    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.139007    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.139013    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139074    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139079    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139083    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139086    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139089    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139092    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139095    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.141867 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139098    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139101    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139104    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139107    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139109    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139113    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139116    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139118    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139121    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139124    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139127    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139130    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139132    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139135    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139137    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139141    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139144    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139146    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139149    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.142250 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139152    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139156    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139161    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139165    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139168    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139171    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139174    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139177    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139179    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139182    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139184    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139187    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139189    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139192    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139194    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139197    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139200    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139202    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139205    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.142708 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139208    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139211    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139213    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139216    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139218    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139221    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139223    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139226    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139228    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139231    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139233    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139236    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139238    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139241    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139243    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139246    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139249    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139252    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139254    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139257    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.143186 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139259    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139262    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139264    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139267    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139269    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139272    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139274    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139277    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139280    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139282    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139285    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139287    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139290    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139293    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139295    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139298    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139301    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139304    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139306    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139309    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.143674 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139311    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139314    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139316    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139320    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139322    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139326    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139330    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139333    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139335    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139339    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139343    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139346    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139349    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139351    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.144315 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.139357    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139474    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139478    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139481    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139485    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139488    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139491    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139495    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139498    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139501    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139504    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139506    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139510    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139512    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139515    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139518    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139520    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139523    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139526    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139529    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 21:50:58.144670 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139531    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139534    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139536    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139538    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139541    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139544    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139547    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139549    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139552    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139555    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139557    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139560    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139562    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139565    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139567    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139570    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139573    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139575    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139578    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139580    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 21:50:58.145170 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139584    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139587    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139590    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139594    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139597    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139600    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139603    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139606    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139609    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139611    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139614    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139617    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139619    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139622    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139624    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139627    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139629    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139632    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139635    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139637    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 21:50:58.145648 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139640    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139642    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139644    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139647    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139649    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139652    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139654    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139657    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139659    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139661    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139664    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139666    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139669    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139671    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139674    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139676    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139679    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139681    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139684    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139687    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 21:50:58.146144 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139689    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139692    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139694    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139698    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139701    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139704    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139706    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139709    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139712    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139714    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139717    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139720    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139722    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139724    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139727    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139729    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139732    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139734    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139737    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 21:50:58.146626 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:58.139739    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.139758    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.139892    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.141867    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.143734    2570 server.go:1019] "Starting client certificate rotation"
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.143849    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.143886    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 21:50:58.147173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.146702    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 21:50:58.152231 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.152208    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 21:50:58.158290 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.158269    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 14 21:50:58.160416 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.160399    2570 log.go:25] "Validated CRI v1 image API"
Jul 14 21:50:58.161199 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.161175    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 21:50:58.163591 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.163565    2570 fs.go:135] Filesystem UUIDs: map[2b7246d4-ab5b-47b1-8c8c-baefa21f7d0a:/dev/nvme0n1p4 55aef622-dab7-477f-9131-892e10f44dcf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 14 21:50:58.163702 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.163588    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 21:50:58.169086 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.168947    2570 manager.go:216] Machine: {Timestamp:2026-07-14 21:50:58.167440464 +0000 UTC m=+0.365474744 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098616 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c4878763784e30e3a8bd3c69e8afc SystemUUID:ec2c4878-7637-84e3-0e3a-8bd3c69e8afc BootID:4b150a9d-add0-4545-993a-3927b2f8db7a Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:49:f6:f6:b7:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:49:f6:f6:b7:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:9d:9a:f4:e3:dc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 21:50:58.169086 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.169070    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 21:50:58.169292 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.169197    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 21:50:58.172854 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.172824    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 21:50:58.173008 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.172857    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 21:50:58.173581 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.173567    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 21:50:58.173619 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.173585    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 21:50:58.173619 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.173593    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 21:50:58.174089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.174078    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 21:50:58.175304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.175292    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 21:50:58.178500 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.178484    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 21:50:58.178633 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.178623    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 21:50:58.179280 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179270    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 21:50:58.179319 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179284    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 21:50:58.179319 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179299    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 21:50:58.179319 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179309    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 14 21:50:58.179319 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179319    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 21:50:58.179837 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179823    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 21:50:58.179907 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.179847    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 21:50:58.181948 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.181924    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 21:50:58.182639 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.182613    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 21:50:58.185538 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.185517    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 21:50:58.185630 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.185557    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 21:50:58.186466 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186451    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 21:50:58.186556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186475    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 21:50:58.186556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186500    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 21:50:58.186556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186526    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 21:50:58.186556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186542    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 21:50:58.186556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.186550    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 21:50:58.187051 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187040    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 21:50:58.187089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187051    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 21:50:58.187089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187061    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 21:50:58.187089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187069    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 21:50:58.187089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187078    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 21:50:58.187089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187088    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 21:50:58.187261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187115    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 21:50:58.187261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.187123    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 21:50:58.191521 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.191500    2570 server.go:1268] "Started kubelet"
Jul 14 21:50:58.191650 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.191592    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 21:50:58.191697 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.191608    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 21:50:58.191735 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.191712    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 21:50:58.192519 ip-10-0-130-163 systemd[1]: Started Kubernetes Kubelet.
Jul 14 21:50:58.193180 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.193006    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 21:50:58.193655 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.193642    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 21:50:58.195603 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.195581    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 21:50:58.195603 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.195595    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 21:50:58.195879 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.195861    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 21:50:58.197283 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.197266    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 21:50:58.200904 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.200789    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 21:50:58.200904 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.200803    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 21:50:58.200904 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.200869    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.201418 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.198196    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18c246721567a6e9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-07-14 21:50:58.191468265 +0000 UTC m=+0.389502541,LastTimestamp:2026-07-14 21:50:58.191468265 +0000 UTC m=+0.389502541,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Jul 14 21:50:58.201418 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.201410    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 21:50:58.201572 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.201427    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 21:50:58.201572 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.201408    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 21:50:58.201572 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.201568    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 21:50:58.201572 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.201576    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 21:50:58.202226 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202127    2570 factory.go:55] Registering systemd factory
Jul 14 21:50:58.202226 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202139    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 14 21:50:58.202566 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202546    2570 factory.go:153] Registering CRI-O factory
Jul 14 21:50:58.202566 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202569    2570 factory.go:223] Registration of the crio container factory successfully
Jul 14 21:50:58.202702 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202625    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 21:50:58.202702 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202648    2570 factory.go:103] Registering Raw factory
Jul 14 21:50:58.202702 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.202663    2570 manager.go:1196] Started watching for new ooms in manager
Jul 14 21:50:58.203393 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.203372    2570 manager.go:349] Starting recovery of all containers
Jul 14 21:50:58.204933 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.204909    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 21:50:58.205076 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.205057    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 21:50:58.212766 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.212569    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-thcqf"
Jul 14 21:50:58.214383 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.214349    2570 manager.go:354] Recovery completed
Jul 14 21:50:58.217602 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.217581    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-thcqf"
Jul 14 21:50:58.219204 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.219192    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.228206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228186    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.228298 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228222    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.228298 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228235    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.228783 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228767    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 21:50:58.228783 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228782    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 21:50:58.228888 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228784    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.228888 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.228800    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 21:50:58.229881 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.229864    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.229977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.229898    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.229977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.229913    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.230073 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.230009    2570 policy_none.go:49] "None policy: Start"
Jul 14 21:50:58.230073 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.230022    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 21:50:58.230073 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.230032    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 21:50:58.230823 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.230718    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-163.ec2.internal.18c24672179835fe  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-163.ec2.internal,UID:ip-10-0-130-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-163.ec2.internal,},FirstTimestamp:2026-07-14 21:50:58.228205054 +0000 UTC m=+0.426239330,LastTimestamp:2026-07-14 21:50:58.228205054 +0000 UTC m=+0.426239330,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-163.ec2.internal,}"
Jul 14 21:50:58.231180 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.231168    2570 policy_none.go:47] "Start"
Jul 14 21:50:58.268508 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.268486    2570 manager.go:337] "Starting Device Plugin manager"
Jul 14 21:50:58.268640 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.268528    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 21:50:58.268640 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.268541    2570 server.go:87] "Starting device plugin registration server"
Jul 14 21:50:58.268863 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.268849    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 21:50:58.268934 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.268864    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 21:50:58.268991 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.268966    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 21:50:58.269088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.269072    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 21:50:58.269170 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.269089    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 21:50:58.269575 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.269556    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 21:50:58.269649 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.269600    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.357481 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.357443    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 21:50:58.359445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.358709    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 21:50:58.359445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.358759    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 21:50:58.359445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.358783    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 21:50:58.359445 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.358826    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 21:50:58.361243 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.361216    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:50:58.369404 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.369383    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.370299 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.370284    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.370374 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.370315    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.370374 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.370326    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.370374 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.370350    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.378686 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.378637    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.378686 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.378662    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-163.ec2.internal\": node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.396144 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.396117    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.459506 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.459469    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"]
Jul 14 21:50:58.459670 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.459553    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.460504 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.460484    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.460599 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.460517    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.460599 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.460529    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.462655 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.462642    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.462779 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.462765    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.462816 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.462798    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.463444 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.463421    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.463545 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.463457    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.463545 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.463470    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.464013 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.463997    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.464097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.464029    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.464097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.464044    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.465441 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.465424    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.465528 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.465451    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 21:50:58.466143 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.466127    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 21:50:58.466209 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.466167    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 21:50:58.466209 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.466181    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeHasSufficientPID"
Jul 14 21:50:58.486665 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.486638    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-163.ec2.internal\" not found" node="ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.491285 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.491264    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-163.ec2.internal\" not found" node="ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.496326 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.496309    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.596407 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.596372    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.603484 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.603452    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d13ebaf0952296d4b090313385ef7786-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"d13ebaf0952296d4b090313385ef7786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.603590 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.603487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.603590 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.603509    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.697262 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.697196    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.704587 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704561    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d13ebaf0952296d4b090313385ef7786-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"d13ebaf0952296d4b090313385ef7786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.704646 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704592    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.704646 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.704707 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704649    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d13ebaf0952296d4b090313385ef7786-config\") pod \"kube-apiserver-proxy-ip-10-0-130-163.ec2.internal\" (UID: \"d13ebaf0952296d4b090313385ef7786\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.704707 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704648    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.704779 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.704709    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ef283a0bd844eb74a25aec9c0349c35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal\" (UID: \"7ef283a0bd844eb74a25aec9c0349c35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.788825 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.788783    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.794552 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:58.794532    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:58.797688 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.797664    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.897852 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.897812    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:58.998428 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:58.998323    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:59.012795 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.012763    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:50:59.099190 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:59.099158    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:59.142757 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.142705    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 21:50:59.143283 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.142891    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 21:50:59.143283 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.142891    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 21:50:59.199821 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:59.199787    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:59.221262 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.221221    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 21:45:59 +0000 UTC" deadline="2028-04-16 13:48:11.629215208 +0000 UTC"
Jul 14 21:50:59.221262 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.221255    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15399h57m12.407963693s"
Jul 14 21:50:59.300006 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:59.299915    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:59.400556 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:50:59.400523    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-163.ec2.internal\" not found"
Jul 14 21:50:59.400767 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.400713    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:50:59.470832 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.470799    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:50:59.501427 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.501390    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:59.513631 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:59.513595    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13ebaf0952296d4b090313385ef7786.slice/crio-578f3c96c73d068b42b23c72127a139b0b49eb9be17f0eaf6e0ea1d7cc9db861 WatchSource:0}: Error finding container 578f3c96c73d068b42b23c72127a139b0b49eb9be17f0eaf6e0ea1d7cc9db861: Status 404 returned error can't find the container with id 578f3c96c73d068b42b23c72127a139b0b49eb9be17f0eaf6e0ea1d7cc9db861
Jul 14 21:50:59.514473 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:50:59.514439    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ef283a0bd844eb74a25aec9c0349c35.slice/crio-4df4840ad5bcbdd41b5a48c723a7a722f18f6e836779dcf23ecd8b6b20ff2278 WatchSource:0}: Error finding container 4df4840ad5bcbdd41b5a48c723a7a722f18f6e836779dcf23ecd8b6b20ff2278: Status 404 returned error can't find the container with id 4df4840ad5bcbdd41b5a48c723a7a722f18f6e836779dcf23ecd8b6b20ff2278
Jul 14 21:50:59.514910 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.514888    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 21:50:59.515891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.515875    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal"
Jul 14 21:50:59.520618 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.519167    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 21:50:59.533227 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:50:59.533204    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 21:51:01.690407 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.690290    2570 apiserver.go:52] "Watching apiserver"
Jul 14 21:51:01.697936 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.697532    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 21:51:01.699096 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.699061    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-55sxh","kube-system/konnectivity-agent-87xhf","openshift-dns/node-resolver-hnntz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal","openshift-multus/multus-additional-cni-plugins-gnrqt","openshift-network-operator/iptables-alerter-7ldtv","kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf","openshift-cluster-node-tuning-operator/tuned-jgwtb","openshift-image-registry/node-ca-2z86g","openshift-multus/multus-m6ztp","openshift-multus/network-metrics-daemon-t9h8k","openshift-network-diagnostics/network-check-target-mk8w6","openshift-ovn-kubernetes/ovnkube-node-jdvlx"]
Jul 14 21:51:01.702261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.701790    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:01.702261 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.701899    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:01.704543 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.704098    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.707019 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.706992    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zk6j6\""
Jul 14 21:51:01.709183 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.707281    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.709183 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.707409    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 21:51:01.709183 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.707629    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 21:51:01.709183 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.708959    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.711621 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.711332    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.711850 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.711828    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.712840 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712009    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 21:51:01.712840 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712086    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.712840 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712496    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gdxrb\""
Jul 14 21:51:01.712840 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712762    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.713088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712843    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 21:51:01.713088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712896    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.713088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.712993    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n6wxj\""
Jul 14 21:51:01.713576 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.713477    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.713715 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.713698    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.714138 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.714116    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fstfb\""
Jul 14 21:51:01.717882 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.716537    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.717882 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.717744    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 21:51:01.717882 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.717825    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 21:51:01.718131 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.717743    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n79fv\""
Jul 14 21:51:01.718131 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.718111    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.718589 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.718567    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 21:51:01.719109 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.718914    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.719109 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.718929    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 21:51:01.719109 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.718989    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.719109 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.719067    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:01.719394 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.719224    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 21:51:01.720735 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.720715    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.721121    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6bdc7\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.721339    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.721539    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.721850    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.722238    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.722539    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.722828 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.722608    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-systemd-units\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723071    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723099    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsscn\" (UniqueName: \"kubernetes.io/projected/7ccf0eb0-d809-494f-9a00-998e2c04af26-kube-api-access-zsscn\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723143    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7726552b-fb3f-4890-bf3e-36a0fc97a326-iptables-alerter-script\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723168    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-slash\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723194    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-systemd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723222 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723217    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-etc-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723241    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38e2c79d-8955-4bfe-b7ff-c671bff1a118-agent-certs\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723327    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723364    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-env-overrides\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723389    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19a004c4-cced-4841-9e55-bee9abe6d81c-host\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723413    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk57b\" (UniqueName: \"kubernetes.io/projected/19a004c4-cced-4841-9e55-bee9abe6d81c-kube-api-access-qk57b\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723439    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-ovn\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723463    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-log-socket\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.723540 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ccf0eb0-d809-494f-9a00-998e2c04af26-hosts-file\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723543    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfw9d\" (UniqueName: \"kubernetes.io/projected/7726552b-fb3f-4890-bf3e-36a0fc97a326-kube-api-access-wfw9d\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723570    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-kubelet\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723615    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723652    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-bin\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723683    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7726552b-fb3f-4890-bf3e-36a0fc97a326-host-slash\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723719    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-netns\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723743    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-var-lib-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723768    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovn-node-metrics-cert\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723791    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnc8l\" (UniqueName: \"kubernetes.io/projected/7d1cf725-a7b1-4a1c-989c-085b2a031405-kube-api-access-bnc8l\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723832    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ccf0eb0-d809-494f-9a00-998e2c04af26-tmp-dir\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723877    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-node-log\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-netd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723924    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-config\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723946    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19a004c4-cced-4841-9e55-bee9abe6d81c-serviceca\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723967    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38e2c79d-8955-4bfe-b7ff-c671bff1a118-konnectivity-ca\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.724004 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.723995    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.724020    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-script-lib\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.724738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.724066    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.727871 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.726560    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.729145 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.729113    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 21:51:01.729271 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.729201    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.731640 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.730864    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 21:51:01.737849 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.737820    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.738155 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.738138    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 21:51:01.738256 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.738211    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-58c7n\""
Jul 14 21:51:01.738316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.738277    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mwdtm\""
Jul 14 21:51:01.738524 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.738507    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.739066 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.738890    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kq9lm\""
Jul 14 21:51:01.739066 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.739056    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:01.739757 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.739187    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:01.745925 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.745895    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 21:51:01.769312 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.769281    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v88tt"
Jul 14 21:51:01.777690 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.777662    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v88tt"
Jul 14 21:51:01.812659 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.812628    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 21:51:01.825008 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.824973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-multus-certs\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825021    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-kubelet-config\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825053    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4vn2\" (UniqueName: \"kubernetes.io/projected/72336e11-b3fb-4935-a795-4bba7cfa7de8-kube-api-access-c4vn2\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825111    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-config\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19a004c4-cced-4841-9e55-bee9abe6d81c-serviceca\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-os-release\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825204    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-netns\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825252    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38e2c79d-8955-4bfe-b7ff-c671bff1a118-konnectivity-ca\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825276    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-sys\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825319    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-tmp\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825400    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-system-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825426    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-cnibin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825471    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825522    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825546    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-run\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825611    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-daemon-config\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825762    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/19a004c4-cced-4841-9e55-bee9abe6d81c-serviceca\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825761    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-registration-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825869    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsscn\" (UniqueName: \"kubernetes.io/projected/7ccf0eb0-d809-494f-9a00-998e2c04af26-kube-api-access-zsscn\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.825891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825876    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-config\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-etc-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825929    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/38e2c79d-8955-4bfe-b7ff-c671bff1a118-konnectivity-ca\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825956    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-host\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825956    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.825992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-os-release\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826009    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-etc-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826023    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826056    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826196    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19a004c4-cced-4841-9e55-bee9abe6d81c-host\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826226    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-var-lib-kubelet\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826234    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826258    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-tuned\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826275    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/19a004c4-cced-4841-9e55-bee9abe6d81c-host\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826283    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-kubelet\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826311    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-system-cni-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826352    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysconfig\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826396    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-conf\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826428    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-bin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826465    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-ovn\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826488    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-log-socket\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826515    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ccf0eb0-d809-494f-9a00-998e2c04af26-hosts-file\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826541    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-ovn\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826596    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-kubelet\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826622    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-log-socket\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.826675 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826651    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ccf0eb0-d809-494f-9a00-998e2c04af26-hosts-file\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826703    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-kubelet\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826738    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnc8l\" (UniqueName: \"kubernetes.io/projected/7d1cf725-a7b1-4a1c-989c-085b2a031405-kube-api-access-bnc8l\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826818    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826846    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7726552b-fb3f-4890-bf3e-36a0fc97a326-host-slash\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826910    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-netns\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826912    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7726552b-fb3f-4890-bf3e-36a0fc97a326-host-slash\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826945    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-var-lib-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826982    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-run-netns\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.826993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-kubernetes\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827016    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-var-lib-openvswitch\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827061    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827096    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdzgv\" (UniqueName: \"kubernetes.io/projected/23db57f2-1def-41e3-8e10-ed4c2297f176-kube-api-access-wdzgv\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827125    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-device-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.827389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ccf0eb0-d809-494f-9a00-998e2c04af26-tmp-dir\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827219    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-node-log\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827241    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-netd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827274    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-cnibin\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827304    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-sys-fs\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827333    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-script-lib\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827358    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-multus\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827422    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-conf-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827456    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-dbus\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827472    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ccf0eb0-d809-494f-9a00-998e2c04af26-tmp-dir\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827480    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-socket-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827503    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-systemd-units\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827518    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-lib-modules\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827548    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-netd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827590    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkhq4\" (UniqueName: \"kubernetes.io/projected/f83f33a6-9973-46b7-91a6-c4731ced428e-kube-api-access-kkhq4\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827616    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-socket-dir-parent\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827645    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7726552b-fb3f-4890-bf3e-36a0fc97a326-iptables-alerter-script\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.828148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827672    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-slash\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827697    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-systemd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827708    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-systemd-units\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827758    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827769    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-node-log\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38e2c79d-8955-4bfe-b7ff-c671bff1a118-agent-certs\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827827    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-env-overrides\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk57b\" (UniqueName: \"kubernetes.io/projected/19a004c4-cced-4841-9e55-bee9abe6d81c-kube-api-access-qk57b\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827880    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-modprobe-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827903    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-systemd\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827931    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-cni-binary-copy\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.827962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828051    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828076    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-k8s-cni-cncf-io\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-etc-kubernetes\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828133    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sb2h\" (UniqueName: \"kubernetes.io/projected/fdc7e983-6358-4474-9304-02e33c48c692-kube-api-access-5sb2h\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828151    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovnkube-script-lib\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.828938 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828162    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfw9d\" (UniqueName: \"kubernetes.io/projected/7726552b-fb3f-4890-bf3e-36a0fc97a326-kube-api-access-wfw9d\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828201    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phxwr\" (UniqueName: \"kubernetes.io/projected/62b38ed6-77ca-4de4-b410-ef26809a2263-kube-api-access-phxwr\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-bin\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovn-node-metrics-cert\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-hostroot\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828411    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-run-systemd\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828646    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-cni-bin\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7d1cf725-a7b1-4a1c-989c-085b2a031405-host-slash\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.828973    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.829038    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7726552b-fb3f-4890-bf3e-36a0fc97a326-iptables-alerter-script\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.829714 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.829195    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7d1cf725-a7b1-4a1c-989c-085b2a031405-env-overrides\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.832967 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.832938    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7d1cf725-a7b1-4a1c-989c-085b2a031405-ovn-node-metrics-cert\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.833105 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.833050    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/38e2c79d-8955-4bfe-b7ff-c671bff1a118-agent-certs\") pod \"konnectivity-agent-87xhf\" (UID: \"38e2c79d-8955-4bfe-b7ff-c671bff1a118\") " pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:01.837001 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.836922    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:01.837001 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.836956    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:01.837001 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.836972    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:01.837252 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.837050    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:02.337026275 +0000 UTC m=+3.025002362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:01.839967 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.839936    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsscn\" (UniqueName: \"kubernetes.io/projected/7ccf0eb0-d809-494f-9a00-998e2c04af26-kube-api-access-zsscn\") pod \"node-resolver-hnntz\" (UID: \"7ccf0eb0-d809-494f-9a00-998e2c04af26\") " pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:01.841161 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.841129    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnc8l\" (UniqueName: \"kubernetes.io/projected/7d1cf725-a7b1-4a1c-989c-085b2a031405-kube-api-access-bnc8l\") pod \"ovnkube-node-jdvlx\" (UID: \"7d1cf725-a7b1-4a1c-989c-085b2a031405\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:01.841548 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.841520    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfw9d\" (UniqueName: \"kubernetes.io/projected/7726552b-fb3f-4890-bf3e-36a0fc97a326-kube-api-access-wfw9d\") pod \"iptables-alerter-7ldtv\" (UID: \"7726552b-fb3f-4890-bf3e-36a0fc97a326\") " pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:01.842289 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.842265    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk57b\" (UniqueName: \"kubernetes.io/projected/19a004c4-cced-4841-9e55-bee9abe6d81c-kube-api-access-qk57b\") pod \"node-ca-2z86g\" (UID: \"19a004c4-cced-4841-9e55-bee9abe6d81c\") " pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:01.876910 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.876767    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"7ef283a0bd844eb74a25aec9c0349c35","Type":"ContainerStarted","Data":"4df4840ad5bcbdd41b5a48c723a7a722f18f6e836779dcf23ecd8b6b20ff2278"}
Jul 14 21:51:01.878608 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.878571    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" event={"ID":"d13ebaf0952296d4b090313385ef7786","Type":"ContainerStarted","Data":"578f3c96c73d068b42b23c72127a139b0b49eb9be17f0eaf6e0ea1d7cc9db861"}
Jul 14 21:51:01.912253 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.912219    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:51:01.928738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.928738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928743    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-device-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-cnibin\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-sys-fs\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-multus\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928858    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-conf-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-dbus\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928908    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-socket-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928935    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-cnibin\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928945    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.928958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-conf-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928970    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-device-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.928993    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-lib-modules\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929008    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-multus\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929017    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-sys-fs\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929020    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkhq4\" (UniqueName: \"kubernetes.io/projected/f83f33a6-9973-46b7-91a6-c4731ced428e-kube-api-access-kkhq4\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929088    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-dbus\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929080    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-socket-dir-parent\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929112    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-lib-modules\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929123    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929136    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-socket-dir-parent\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-modprobe-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929172    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-systemd\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929200    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-cni-binary-copy\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929206    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-socket-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929211    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-systemd\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929224    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.929426 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929265    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-modprobe-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929267    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.929303    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929311    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-k8s-cni-cncf-io\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929342    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-etc-kubernetes\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929347    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-etc-selinux\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.929362    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:02.429342728 +0000 UTC m=+3.117318802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929391    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-etc-kubernetes\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929396    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-k8s-cni-cncf-io\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929435    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sb2h\" (UniqueName: \"kubernetes.io/projected/fdc7e983-6358-4474-9304-02e33c48c692-kube-api-access-5sb2h\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929462    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phxwr\" (UniqueName: \"kubernetes.io/projected/62b38ed6-77ca-4de4-b410-ef26809a2263-kube-api-access-phxwr\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929487    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929512    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-hostroot\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929536    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-multus-certs\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-kubelet-config\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4vn2\" (UniqueName: \"kubernetes.io/projected/72336e11-b3fb-4935-a795-4bba7cfa7de8-kube-api-access-c4vn2\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929612    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-os-release\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.930318 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929637    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-netns\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929661    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-sys\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929710    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-tmp\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929733    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-system-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-cnibin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929782    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929837    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-run\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929852    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-cni-binary-copy\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-daemon-config\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929920    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-system-cni-dir\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929926    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-hostroot\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.929984    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-sys\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-registration-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930040    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-host\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930065    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-os-release\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930117    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-var-lib-kubelet\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931187 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930139    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-tuned\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-kubelet\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-system-cni-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930207    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysconfig\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930231    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-conf\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-bin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930314    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930334    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62b38ed6-77ca-4de4-b410-ef26809a2263-multus-daemon-config\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930340    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-kubernetes\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930411    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-kubernetes\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-cnibin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930438    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930471    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdzgv\" (UniqueName: \"kubernetes.io/projected/23db57f2-1def-41e3-8e10-ed4c2297f176-kube-api-access-wdzgv\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930611    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-run\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930744    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-var-lib-kubelet\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930892    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-conf\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.931981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-cni-bin\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.930981    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.931062    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931067    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysctl-d\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931135    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/23db57f2-1def-41e3-8e10-ed4c2297f176-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:01.931159    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:02.431140986 +0000 UTC m=+3.119117078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931199    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-multus-certs\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931243    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931247    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/385bbb00-e382-4a0b-927b-13a51ee2d1d3-kubelet-config\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931265    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-os-release\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931295    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-var-lib-kubelet\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931314    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-host\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931361    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/72336e11-b3fb-4935-a795-4bba7cfa7de8-registration-dir\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931354    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-os-release\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62b38ed6-77ca-4de4-b410-ef26809a2263-host-run-netns\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931410    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/23db57f2-1def-41e3-8e10-ed4c2297f176-system-cni-dir\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.932835 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.931501    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-sysconfig\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.933501 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.932918    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-tmp\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.934145 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.934117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f83f33a6-9973-46b7-91a6-c4731ced428e-etc-tuned\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.943561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.943482    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sb2h\" (UniqueName: \"kubernetes.io/projected/fdc7e983-6358-4474-9304-02e33c48c692-kube-api-access-5sb2h\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:01.944020 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.943992    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkhq4\" (UniqueName: \"kubernetes.io/projected/f83f33a6-9973-46b7-91a6-c4731ced428e-kube-api-access-kkhq4\") pod \"tuned-jgwtb\" (UID: \"f83f33a6-9973-46b7-91a6-c4731ced428e\") " pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:01.944785 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.944463    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdzgv\" (UniqueName: \"kubernetes.io/projected/23db57f2-1def-41e3-8e10-ed4c2297f176-kube-api-access-wdzgv\") pod \"multus-additional-cni-plugins-gnrqt\" (UID: \"23db57f2-1def-41e3-8e10-ed4c2297f176\") " pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:01.944912 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.944891    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4vn2\" (UniqueName: \"kubernetes.io/projected/72336e11-b3fb-4935-a795-4bba7cfa7de8-kube-api-access-c4vn2\") pod \"aws-ebs-csi-driver-node-gvvvf\" (UID: \"72336e11-b3fb-4935-a795-4bba7cfa7de8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:01.945531 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:01.945502    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phxwr\" (UniqueName: \"kubernetes.io/projected/62b38ed6-77ca-4de4-b410-ef26809a2263-kube-api-access-phxwr\") pod \"multus-m6ztp\" (UID: \"62b38ed6-77ca-4de4-b410-ef26809a2263\") " pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:02.019226 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.019185    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:02.036552 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.036507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hnntz"
Jul 14 21:51:02.051653 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.051608    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ldtv"
Jul 14 21:51:02.060822 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.060774    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:02.067948 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.067910    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z86g"
Jul 14 21:51:02.077994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.077952    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m6ztp"
Jul 14 21:51:02.087955 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.087913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnrqt"
Jul 14 21:51:02.097995 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.097958    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf"
Jul 14 21:51:02.105014 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.104983    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jgwtb"
Jul 14 21:51:02.434615 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.434526    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:02.434615 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.434573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:02.434615 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.434608    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434726    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434798    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434736    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434833    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:03.434796214 +0000 UTC m=+4.122772300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434852    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:03.434840381 +0000 UTC m=+4.122816477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434851    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:02.434890 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434879    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:02.435133 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.434933    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:03.434918489 +0000 UTC m=+4.122894583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:02.781261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.781159    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 21:46:01 +0000 UTC" deadline="2028-02-04 11:53:55.304433361 +0000 UTC"
Jul 14 21:51:02.781261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.781201    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13670h2m52.523238088s"
Jul 14 21:51:02.869933 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:02.869898    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:02.870105 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:02.870018    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:03.005432 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.005392    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e2c79d_8955_4bfe_b7ff_c671bff1a118.slice/crio-7ea0ab6833da29b35b2a9daf3621e04f7df468f07676ee57335ac6e5a42eb17a WatchSource:0}: Error finding container 7ea0ab6833da29b35b2a9daf3621e04f7df468f07676ee57335ac6e5a42eb17a: Status 404 returned error can't find the container with id 7ea0ab6833da29b35b2a9daf3621e04f7df468f07676ee57335ac6e5a42eb17a
Jul 14 21:51:03.009760 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.009726    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1cf725_a7b1_4a1c_989c_085b2a031405.slice/crio-7e8e1522f4e47791677d33f8cc6ab4b92b28ac1cd56d5b038cb1c66edfdfb913 WatchSource:0}: Error finding container 7e8e1522f4e47791677d33f8cc6ab4b92b28ac1cd56d5b038cb1c66edfdfb913: Status 404 returned error can't find the container with id 7e8e1522f4e47791677d33f8cc6ab4b92b28ac1cd56d5b038cb1c66edfdfb913
Jul 14 21:51:03.012182 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.012151    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a004c4_cced_4841_9e55_bee9abe6d81c.slice/crio-004adcaaa8e41d7428d060bc7fad2be083489ce758b83ef07dc90b300cec2884 WatchSource:0}: Error finding container 004adcaaa8e41d7428d060bc7fad2be083489ce758b83ef07dc90b300cec2884: Status 404 returned error can't find the container with id 004adcaaa8e41d7428d060bc7fad2be083489ce758b83ef07dc90b300cec2884
Jul 14 21:51:03.013897 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.013869    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83f33a6_9973_46b7_91a6_c4731ced428e.slice/crio-be36c18fc2d1a9620aef9e3629aeec1d94b5c785041f5f5c3c572230cd39ed87 WatchSource:0}: Error finding container be36c18fc2d1a9620aef9e3629aeec1d94b5c785041f5f5c3c572230cd39ed87: Status 404 returned error can't find the container with id be36c18fc2d1a9620aef9e3629aeec1d94b5c785041f5f5c3c572230cd39ed87
Jul 14 21:51:03.035248 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.035221    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b38ed6_77ca_4de4_b410_ef26809a2263.slice/crio-1301b511677543615998aa331fcca6704d20e0f456efe9f56d5d502695349b08 WatchSource:0}: Error finding container 1301b511677543615998aa331fcca6704d20e0f456efe9f56d5d502695349b08: Status 404 returned error can't find the container with id 1301b511677543615998aa331fcca6704d20e0f456efe9f56d5d502695349b08
Jul 14 21:51:03.036143 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.036113    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23db57f2_1def_41e3_8e10_ed4c2297f176.slice/crio-c1eaacc6ed5e30f1f3bd7e297cb4d4a4a63284e0fa4e2fc25a3daef0c52236dc WatchSource:0}: Error finding container c1eaacc6ed5e30f1f3bd7e297cb4d4a4a63284e0fa4e2fc25a3daef0c52236dc: Status 404 returned error can't find the container with id c1eaacc6ed5e30f1f3bd7e297cb4d4a4a63284e0fa4e2fc25a3daef0c52236dc
Jul 14 21:51:03.037457 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.037010    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccf0eb0_d809_494f_9a00_998e2c04af26.slice/crio-708700a518bb5f1741205bb552629fcd6af2bdb048d333d0ea4910c146196c89 WatchSource:0}: Error finding container 708700a518bb5f1741205bb552629fcd6af2bdb048d333d0ea4910c146196c89: Status 404 returned error can't find the container with id 708700a518bb5f1741205bb552629fcd6af2bdb048d333d0ea4910c146196c89
Jul 14 21:51:03.037861 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.037840    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7726552b_fb3f_4890_bf3e_36a0fc97a326.slice/crio-16418242b17486b2b1a9ce3fe5d5872f0e439b670d421666e24571b5622273b7 WatchSource:0}: Error finding container 16418242b17486b2b1a9ce3fe5d5872f0e439b670d421666e24571b5622273b7: Status 404 returned error can't find the container with id 16418242b17486b2b1a9ce3fe5d5872f0e439b670d421666e24571b5622273b7
Jul 14 21:51:03.039772 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:03.039736    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72336e11_b3fb_4935_a795_4bba7cfa7de8.slice/crio-da1ed2aa4151a414bff418abfb171967d2382e4391445ea09f88aa27916e9146 WatchSource:0}: Error finding container da1ed2aa4151a414bff418abfb171967d2382e4391445ea09f88aa27916e9146: Status 404 returned error can't find the container with id da1ed2aa4151a414bff418abfb171967d2382e4391445ea09f88aa27916e9146
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.441602    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.441672    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.441705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.441894    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.441918    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.441931    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.441994    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:05.441974788 +0000 UTC m=+6.129950870 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.442071    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.442103    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:05.442091849 +0000 UTC m=+6.130067932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.442150    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:03.442294 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.442185    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:05.442175388 +0000 UTC m=+6.130151465 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:03.687934 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.687742    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 21:51:03.781617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.781418    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 21:46:01 +0000 UTC" deadline="2027-12-25 07:53:25.518091002 +0000 UTC"
Jul 14 21:51:03.781617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.781477    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12682h2m21.73661845s"
Jul 14 21:51:03.872072 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.871527    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:03.872072 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.871674    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:03.872311 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.872102    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:03.872311 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:03.872222    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:03.906103 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.906063    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" event={"ID":"d13ebaf0952296d4b090313385ef7786","Type":"ContainerStarted","Data":"37576284f5a97c7fb28c13f9fedb221decf223bf1e8c20863a1f80b52728dd30"}
Jul 14 21:51:03.924214 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.924100    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf" event={"ID":"72336e11-b3fb-4935-a795-4bba7cfa7de8","Type":"ContainerStarted","Data":"da1ed2aa4151a414bff418abfb171967d2382e4391445ea09f88aa27916e9146"}
Jul 14 21:51:03.931542 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.931446    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ldtv" event={"ID":"7726552b-fb3f-4890-bf3e-36a0fc97a326","Type":"ContainerStarted","Data":"16418242b17486b2b1a9ce3fe5d5872f0e439b670d421666e24571b5622273b7"}
Jul 14 21:51:03.943483 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.943444    2570 generic.go:356] "Generic (PLEG): container finished" podID="7ef283a0bd844eb74a25aec9c0349c35" containerID="4345d40ba60f3fdb59710e472a077fbc5704b25c017338240cf82f6f1d3d5f43" exitCode=0
Jul 14 21:51:03.943934 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.943835    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"7ef283a0bd844eb74a25aec9c0349c35","Type":"ContainerDied","Data":"4345d40ba60f3fdb59710e472a077fbc5704b25c017338240cf82f6f1d3d5f43"}
Jul 14 21:51:03.953752 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.953716    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnntz" event={"ID":"7ccf0eb0-d809-494f-9a00-998e2c04af26","Type":"ContainerStarted","Data":"708700a518bb5f1741205bb552629fcd6af2bdb048d333d0ea4910c146196c89"}
Jul 14 21:51:03.965412 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.965350    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-163.ec2.internal" podStartSLOduration=2.965326403 podStartE2EDuration="2.965326403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:03.924230999 +0000 UTC m=+4.612207089" watchObservedRunningTime="2026-07-14 21:51:03.965326403 +0000 UTC m=+4.653302497"
Jul 14 21:51:03.972015 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.971973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerStarted","Data":"c1eaacc6ed5e30f1f3bd7e297cb4d4a4a63284e0fa4e2fc25a3daef0c52236dc"}
Jul 14 21:51:03.982369 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.982331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m6ztp" event={"ID":"62b38ed6-77ca-4de4-b410-ef26809a2263","Type":"ContainerStarted","Data":"1301b511677543615998aa331fcca6704d20e0f456efe9f56d5d502695349b08"}
Jul 14 21:51:03.998228 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:03.996381    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jgwtb" event={"ID":"f83f33a6-9973-46b7-91a6-c4731ced428e","Type":"ContainerStarted","Data":"be36c18fc2d1a9620aef9e3629aeec1d94b5c785041f5f5c3c572230cd39ed87"}
Jul 14 21:51:04.001201 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:04.001141    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z86g" event={"ID":"19a004c4-cced-4841-9e55-bee9abe6d81c","Type":"ContainerStarted","Data":"004adcaaa8e41d7428d060bc7fad2be083489ce758b83ef07dc90b300cec2884"}
Jul 14 21:51:04.004405 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:04.004301    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"7e8e1522f4e47791677d33f8cc6ab4b92b28ac1cd56d5b038cb1c66edfdfb913"}
Jul 14 21:51:04.012711 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:04.012508    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87xhf" event={"ID":"38e2c79d-8955-4bfe-b7ff-c671bff1a118","Type":"ContainerStarted","Data":"7ea0ab6833da29b35b2a9daf3621e04f7df468f07676ee57335ac6e5a42eb17a"}
Jul 14 21:51:04.870035 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:04.870001    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:04.870524 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:04.870138    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:05.045047 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.044976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" event={"ID":"7ef283a0bd844eb74a25aec9c0349c35","Type":"ContainerStarted","Data":"93c7d49a61c1aaf1cb636d97e76ebbe15dd7d90f4153192711c75837c51ee91d"}
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.461451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.461509    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.461573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461697    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461760    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:09.461740843 +0000 UTC m=+10.149716935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461850    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461884    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:09.46187279 +0000 UTC m=+10.149848879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461965    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461978    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.461991    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:05.462362 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.462025    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:09.462014711 +0000 UTC m=+10.149990797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:05.870023 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.869933    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:05.870176 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.870088    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:05.870176 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:05.869932    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:05.870578 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:05.870204    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:06.869460 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:06.869423    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:06.869664 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:06.869562    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:07.870266 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:07.870111    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:07.870266 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:07.870140    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:07.870787 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:07.870270    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:07.870787 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:07.870365    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:08.869398 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:08.869361    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:08.869587 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:08.869498    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:09.495674 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:09.495574    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:09.495674 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:09.495643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:09.495683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495757    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495777    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495863    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:17.495840109 +0000 UTC m=+18.183816203 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495884    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:17.495874278 +0000 UTC m=+18.183850363 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495903    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495922    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495933    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:09.496216 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.495995    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:17.495976414 +0000 UTC m=+18.183952504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:09.872790 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:09.872714    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:09.873155 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.873122    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:09.873591 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:09.873570    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:09.873817 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:09.873747    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:10.870132 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:10.869926    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:10.870132 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:10.870080    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:11.870348 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:11.869828    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:11.870348 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:11.869966    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:11.870348 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:11.869828    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:11.870348 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:11.870306    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:12.869223 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:12.869186    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:12.869399 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:12.869322    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:13.869673 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:13.869639    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:13.870123 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:13.869642    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:13.870123 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:13.869769    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:13.870123 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:13.869866    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:14.869097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:14.869047    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:14.869293 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:14.869170    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:15.870188 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:15.870147    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:15.870679 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:15.870200    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:15.870679 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:15.870310    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:15.870679 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:15.870454    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:16.869051 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:16.869009    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:16.869266 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:16.869129    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:17.556988 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:17.556943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:17.557005    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:17.557038    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557097    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557128    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557166    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557177    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:33.557157534 +0000 UTC m=+34.245133622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557184    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557195    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:33.557186046 +0000 UTC m=+34.245162120 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557196    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:17.557406 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.557255    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:51:33.557239884 +0000 UTC m=+34.245215973 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:17.870065 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:17.869987    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:17.870225 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:17.869987    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:17.870225 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.870118    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:17.870316 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:17.870224    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:18.869611 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:18.869571    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:18.870107 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:18.869708    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:19.872369 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:19.872340    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:19.872790 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:19.872352    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:19.872790 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:19.872452    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:19.872790 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:19.872525    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:20.869631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:20.869592    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:20.869849 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:20.869734    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:21.870532 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:21.869149    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:21.870532 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:21.869488    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:21.870532 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:21.869567    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:21.870532 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:21.869637    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:22.082368 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.082109    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hnntz" event={"ID":"7ccf0eb0-d809-494f-9a00-998e2c04af26","Type":"ContainerStarted","Data":"0f8e6e840d7649f9e2a80b8748e1a43cf7b66dcb263111ca194806d5b84af2cf"}
Jul 14 21:51:22.084021 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.083992    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="38bf25a62ed2436913169cbda57b0441a9fba450591281e76d58291983833985" exitCode=0
Jul 14 21:51:22.084182 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.084079    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"38bf25a62ed2436913169cbda57b0441a9fba450591281e76d58291983833985"}
Jul 14 21:51:22.086297 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.086196    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m6ztp" event={"ID":"62b38ed6-77ca-4de4-b410-ef26809a2263","Type":"ContainerStarted","Data":"e204e4a4727a69164c446f599d424bb57fd484ce5d5f24c2384e50e9d3e505df"}
Jul 14 21:51:22.088056 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.088021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jgwtb" event={"ID":"f83f33a6-9973-46b7-91a6-c4731ced428e","Type":"ContainerStarted","Data":"cf8d4b1f516b1092eb584cd1033cea68d50c1ce546ff098cb73fe340da020df3"}
Jul 14 21:51:22.089624 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.089601    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z86g" event={"ID":"19a004c4-cced-4841-9e55-bee9abe6d81c","Type":"ContainerStarted","Data":"06651215691afeadf8094fdb84ee6ee197f2a07f7718d8efe2d041c64fa7997b"}
Jul 14 21:51:22.092661 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.092638    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:51:22.093027 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.093003    2570 generic.go:356] "Generic (PLEG): container finished" podID="7d1cf725-a7b1-4a1c-989c-085b2a031405" containerID="736b27ec9566e0e9196ef5d95a86923d19d4a61ed74f7076305c5c0457d829f7" exitCode=1
Jul 14 21:51:22.093140 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.093071    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"4e326b22927e3f28293f0cc674024a9378fd0700bf8b7d069b7f52a465c048e2"}
Jul 14 21:51:22.093188 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.093142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerDied","Data":"736b27ec9566e0e9196ef5d95a86923d19d4a61ed74f7076305c5c0457d829f7"}
Jul 14 21:51:22.093241 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.093195    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"674add3becf016780d29019405bddcf56dad4b81c97521011fec202d9ed7d272"}
Jul 14 21:51:22.094556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.094529    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-87xhf" event={"ID":"38e2c79d-8955-4bfe-b7ff-c671bff1a118","Type":"ContainerStarted","Data":"4c4cc9ca6ccbf044059f8cfa82c8ca7c1f3cff2c8f429a2c928de202bbd249df"}
Jul 14 21:51:22.096077 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.096050    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf" event={"ID":"72336e11-b3fb-4935-a795-4bba7cfa7de8","Type":"ContainerStarted","Data":"d7f85b5c94e9912b213ea4c27ffc20b180e5a415271db894215df7a38e91b44d"}
Jul 14 21:51:22.099977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.099506    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hnntz" podStartSLOduration=4.460628362 podStartE2EDuration="23.099485351s" totalImagesPullingTime="18.638856989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:22.098914267 +0000 UTC m=+22.786890375" watchObservedRunningTime="2026-07-14 21:51:22.099485351 +0000 UTC m=+22.787461445"
Jul 14 21:51:22.099977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.099606    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-163.ec2.internal" podStartSLOduration=21.099599694 podStartE2EDuration="21.099599694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:05.066776791 +0000 UTC m=+5.754752885" watchObservedRunningTime="2026-07-14 21:51:22.099599694 +0000 UTC m=+22.787575790"
Jul 14 21:51:22.115937 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.115886    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jgwtb" podStartSLOduration=3.473265833 podStartE2EDuration="22.115869931s" totalImagesPullingTime="18.642604098s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:22.115438423 +0000 UTC m=+22.803414516" watchObservedRunningTime="2026-07-14 21:51:22.115869931 +0000 UTC m=+22.803846024"
Jul 14 21:51:22.129729 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.129663    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2z86g" podStartSLOduration=4.487057727 podStartE2EDuration="23.129640037s" totalImagesPullingTime="18.64258231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:22.129376132 +0000 UTC m=+22.817352226" watchObservedRunningTime="2026-07-14 21:51:22.129640037 +0000 UTC m=+22.817616131"
Jul 14 21:51:22.147066 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.147020    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m6ztp" podStartSLOduration=4.445528373 podStartE2EDuration="23.147004882s" totalImagesPullingTime="18.701476509s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:22.146718642 +0000 UTC m=+22.834694734" watchObservedRunningTime="2026-07-14 21:51:22.147004882 +0000 UTC m=+22.834980975"
Jul 14 21:51:22.165450 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.165391    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-87xhf" podStartSLOduration=4.49536701 podStartE2EDuration="23.165368021s" totalImagesPullingTime="18.670001011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:22.164704564 +0000 UTC m=+22.852680676" watchObservedRunningTime="2026-07-14 21:51:22.165368021 +0000 UTC m=+22.853344115"
Jul 14 21:51:22.254261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.254220    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:22.326007 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.325973    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:22.869485 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:22.869442    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:22.869652 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:22.869548    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:23.102866 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.102833    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:51:23.103588 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.103296    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"004d12b94c6a21cec1ddc2c7a210fa7ba3974a495f99708d5c13717f56ee9813"}
Jul 14 21:51:23.103588 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.103390    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"80b8593be4479fe5ab6b1ab1985fb5edc4004badf59e69d7deaf04cd522c4f31"}
Jul 14 21:51:23.103588 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.103443    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"6fdda38fddfb3718232481f0729f568a99709db57a0e66d29cb0474ef23ede61"}
Jul 14 21:51:23.105065 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.105030    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ldtv" event={"ID":"7726552b-fb3f-4890-bf3e-36a0fc97a326","Type":"ContainerStarted","Data":"25e23dd09db11dedc339ca07e85323c4f92c348889b4c35356809ca4cb5dcfaf"}
Jul 14 21:51:23.120933 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.120871    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7ldtv" podStartSLOduration=5.481777888 podStartE2EDuration="24.120848298s" totalImagesPullingTime="18.63907041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:23.120664855 +0000 UTC m=+23.808640985" watchObservedRunningTime="2026-07-14 21:51:23.120848298 +0000 UTC m=+23.808824391"
Jul 14 21:51:23.224920 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.224793    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 21:51:23.244906 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.244869    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:23.347485 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.347346    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:23.793886 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.793749    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T21:51:23.224834032Z","UUID":"3fa221ed-d2a8-40aa-9785-bd8004960536","Handler":null,"Name":"","Endpoint":""}
Jul 14 21:51:23.795797 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.795772    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 21:51:23.795969 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.795851    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 21:51:23.869583 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.869546    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:23.869583 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:23.869581    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:23.869830 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:23.869737    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:23.870129 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:23.870095    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:24.110614 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:24.110482    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf" event={"ID":"72336e11-b3fb-4935-a795-4bba7cfa7de8","Type":"ContainerStarted","Data":"45cdea28345da562b8e6080a1413d207af3a15c91bca8854be547d3225749923"}
Jul 14 21:51:24.321143 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:24.321102    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:24.394511 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:24.394457    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:24.870181 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:24.870005    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:24.870371 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:24.870262    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:25.115854 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:25.115822    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:51:25.116339 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:25.116232    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"be787c671b858e8a9f9f521617e7dde3292dfed7957ca7c65a3c56eea6919625"}
Jul 14 21:51:25.118308 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:25.118280    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf" event={"ID":"72336e11-b3fb-4935-a795-4bba7cfa7de8","Type":"ContainerStarted","Data":"e4fd5e99323186bed9b790a98ebd9a78125bc4a40c112af45cc837e914f6fa6f"}
Jul 14 21:51:25.869626 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:25.869590    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:25.869626 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:25.869637    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:25.869947 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:25.869729    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:25.869947 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:25.869835    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:26.870191 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:26.869865    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:26.870191 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:26.869999    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:26.997694 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:26.997503    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:26.998248 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:26.998228    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:27.013879 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.013825    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gvvvf" podStartSLOduration=6.161222752 podStartE2EDuration="27.013783486s" totalImagesPullingTime="20.852560734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:25.137160163 +0000 UTC m=+25.825136275" watchObservedRunningTime="2026-07-14 21:51:27.013783486 +0000 UTC m=+27.701759609"
Jul 14 21:51:27.126289 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.126264    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:51:27.126664 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.126636    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"3ee052d8101ff4ac4effec56d3b3d9b22e24521bf2011e13a5005c4b7da72fae"}
Jul 14 21:51:27.126910 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.126892    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:27.127097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.127079    2570 scope.go:117] "RemoveContainer" containerID="736b27ec9566e0e9196ef5d95a86923d19d4a61ed74f7076305c5c0457d829f7"
Jul 14 21:51:27.128698 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.128670    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="866e988b58b6f74f4f3297e28d616a8787875beba371b9570b5498d9ed9cfdd7" exitCode=0
Jul 14 21:51:27.128790 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.128766    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"866e988b58b6f74f4f3297e28d616a8787875beba371b9570b5498d9ed9cfdd7"}
Jul 14 21:51:27.146245 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.146218    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:27.869579 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.869492    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:27.869750 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:27.869611    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:27.869750 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:27.869643    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:27.869750 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:27.869693    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:28.135584 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.135497    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:51:28.136110 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.135926    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" event={"ID":"7d1cf725-a7b1-4a1c-989c-085b2a031405","Type":"ContainerStarted","Data":"2338f5025713ae95bfad41219785480ffb5c3050db8fa0ece869177554f2e594"}
Jul 14 21:51:28.136110 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.136053    2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 14 21:51:28.136325 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.136307    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:28.154519 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.154487    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:28.167495 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.166851    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" podStartSLOduration=10.437806093 podStartE2EDuration="29.166830523s" totalImagesPullingTime="18.72902443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:28.165674426 +0000 UTC m=+28.853650538" watchObservedRunningTime="2026-07-14 21:51:28.166830523 +0000 UTC m=+28.854806614"
Jul 14 21:51:28.842558 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.842521    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mk8w6"]
Jul 14 21:51:28.842774 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.842683    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:28.842852 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:28.842766    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:28.845509 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.845479    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t9h8k"]
Jul 14 21:51:28.845637 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.845629    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:28.845760 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:28.845741    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:28.846253 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.846228    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-55sxh"]
Jul 14 21:51:28.846360 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:28.846341    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:28.846466 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:28.846449    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:29.140627 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.140530    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="e0935b11e7d564633fd9ecb3111b659736f6b5ed7fdb5dd015bf75a10f38d642" exitCode=0
Jul 14 21:51:29.140627 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.140612    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"e0935b11e7d564633fd9ecb3111b659736f6b5ed7fdb5dd015bf75a10f38d642"}
Jul 14 21:51:29.141107 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.140726    2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 14 21:51:29.187376 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.187339    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:29.260789 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.260755    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:29.462561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:29.462527    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:30.143617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.143588    2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 14 21:51:30.225357 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.225327    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:30.290219 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.290187    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:30.869727 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.869690    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:30.869930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.869815    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:30.869930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.869846    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:30.869930 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:30.869835    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:30.870082 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:30.869935    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:30.870082 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:30.869989    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:30.998477 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:30.998438    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:51:31.108322 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.108287    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:31.108504 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.108423    2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 14 21:51:31.109025 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.108988    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-87xhf"
Jul 14 21:51:31.148827 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.148775    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="c3c414b39dfd64dd91fec390854606e40ac8ebd73cfe41cb7cfad37db9b24bd1" exitCode=0
Jul 14 21:51:31.149498 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.148840    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"c3c414b39dfd64dd91fec390854606e40ac8ebd73cfe41cb7cfad37db9b24bd1"}
Jul 14 21:51:31.159476 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.159433    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx" podUID="7d1cf725-a7b1-4a1c-989c-085b2a031405" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 14 21:51:31.209288 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.209260    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:31.612210 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.612177    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:31.668568 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:31.668530    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:32.869918 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:32.869889    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:32.869918 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:32.869924    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:32.869918 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:32.869921    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:32.870522 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:32.870024    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t9h8k" podUID="fdc7e983-6358-4474-9304-02e33c48c692"
Jul 14 21:51:32.870522 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:32.870114    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mk8w6" podUID="dcb2eedd-b418-4246-9dac-b4175d983303"
Jul 14 21:51:32.870522 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:32.870213    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-55sxh" podUID="385bbb00-e382-4a0b-927b-13a51ee2d1d3"
Jul 14 21:51:33.576416 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:33.576374    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:33.576592 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:33.576431    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:33.576592 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:33.576467    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:33.576592 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576540    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576592    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576610    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576623    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5qfhw for pod openshift-network-diagnostics/network-check-target-mk8w6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576631    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576611    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs podName:fdc7e983-6358-4474-9304-02e33c48c692 nodeName:}" failed. No retries permitted until 2026-07-14 21:52:05.576592284 +0000 UTC m=+66.264568369 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs") pod "network-metrics-daemon-t9h8k" (UID: "fdc7e983-6358-4474-9304-02e33c48c692") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576664    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw podName:dcb2eedd-b418-4246-9dac-b4175d983303 nodeName:}" failed. No retries permitted until 2026-07-14 21:52:05.576654875 +0000 UTC m=+66.264630945 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qfhw" (UniqueName: "kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw") pod "network-check-target-mk8w6" (UID: "dcb2eedd-b418-4246-9dac-b4175d983303") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 21:51:33.576740 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:33.576697    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret podName:385bbb00-e382-4a0b-927b-13a51ee2d1d3 nodeName:}" failed. No retries permitted until 2026-07-14 21:52:05.576679329 +0000 UTC m=+66.264655413 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret") pod "global-pull-secret-syncer-55sxh" (UID: "385bbb00-e382-4a0b-927b-13a51ee2d1d3") : object "kube-system"/"original-pull-secret" not registered
Jul 14 21:51:34.597503 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.597431    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-163.ec2.internal" event="NodeReady"
Jul 14 21:51:34.598081 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.597594    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 21:51:34.646135 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.646100    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qgrtx"]
Jul 14 21:51:34.677557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.677516    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tlh5c"]
Jul 14 21:51:34.677784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.677662    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.680336 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.680294    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 21:51:34.680481 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.680364    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 21:51:34.680561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.680546    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2wg9m\""
Jul 14 21:51:34.693446 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.693405    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tlh5c"]
Jul 14 21:51:34.693446 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.693452    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qgrtx"]
Jul 14 21:51:34.693646 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.693566    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.695913 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.695706    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f67q8\""
Jul 14 21:51:34.695913 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.695706    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 21:51:34.695913 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.695784    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 21:51:34.695913 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.695903    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 21:51:34.763029 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.762982    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:34.784734 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/322680e0-990d-495b-98d4-70629064e077-cert\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.784994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784757    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9knx9\" (UniqueName: \"kubernetes.io/projected/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-kube-api-access-9knx9\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.784994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784855    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-config-volume\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.784994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784906    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.784994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784956    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65xnm\" (UniqueName: \"kubernetes.io/projected/322680e0-990d-495b-98d4-70629064e077-kube-api-access-65xnm\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.784994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.784993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-tmp-dir\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.869980 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.869900    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:51:34.870157 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.869898    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:51:34.870157 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.869898    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:51:34.873088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873029    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 21:51:34.873088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873073    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 21:51:34.873294 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873099    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 21:51:34.873378 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873361    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 21:51:34.873564 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873549    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5sfp\""
Jul 14 21:51:34.873689 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.873668    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r4xkn\""
Jul 14 21:51:34.885333 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/322680e0-990d-495b-98d4-70629064e077-cert\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.885518 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885352    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9knx9\" (UniqueName: \"kubernetes.io/projected/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-kube-api-access-9knx9\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.885518 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-config-volume\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.885518 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885489    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.885829 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65xnm\" (UniqueName: \"kubernetes.io/projected/322680e0-990d-495b-98d4-70629064e077-kube-api-access-65xnm\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.885829 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885591    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-tmp-dir\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.885974 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.885951    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-tmp-dir\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.885974 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:34.885963    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:34.886704 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.886677    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-config-volume\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.886985 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:34.886966    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:35.38693408 +0000 UTC m=+36.074910164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:34.894602 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.894573    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/322680e0-990d-495b-98d4-70629064e077-cert\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:34.894719 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.894594    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9knx9\" (UniqueName: \"kubernetes.io/projected/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-kube-api-access-9knx9\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:34.897105 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:34.897074    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65xnm\" (UniqueName: \"kubernetes.io/projected/322680e0-990d-495b-98d4-70629064e077-kube-api-access-65xnm\") pod \"ingress-canary-tlh5c\" (UID: \"322680e0-990d-495b-98d4-70629064e077\") " pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:35.004931 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:35.004894    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tlh5c"
Jul 14 21:51:35.320447 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:35.320193    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tlh5c"]
Jul 14 21:51:35.361849 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:35.361799    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod322680e0_990d_495b_98d4_70629064e077.slice/crio-7925650d9b2f2dfc3ecdfe97cee91700ab0084e117c23d1afe403152a38173f2 WatchSource:0}: Error finding container 7925650d9b2f2dfc3ecdfe97cee91700ab0084e117c23d1afe403152a38173f2: Status 404 returned error can't find the container with id 7925650d9b2f2dfc3ecdfe97cee91700ab0084e117c23d1afe403152a38173f2
Jul 14 21:51:35.390137 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:35.390115    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:35.390259 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:35.390233    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:35.390305 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:35.390292    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:36.390274345 +0000 UTC m=+37.078250437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:36.163788 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:36.163746    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tlh5c" event={"ID":"322680e0-990d-495b-98d4-70629064e077","Type":"ContainerStarted","Data":"7925650d9b2f2dfc3ecdfe97cee91700ab0084e117c23d1afe403152a38173f2"}
Jul 14 21:51:36.166948 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:36.166913    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="ff06e2ddedaf645db7844c66bcdc3b3ea7081db44921d3ffa25c949979a889c2" exitCode=0
Jul 14 21:51:36.167096 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:36.166960    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"ff06e2ddedaf645db7844c66bcdc3b3ea7081db44921d3ffa25c949979a889c2"}
Jul 14 21:51:36.398548 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:36.398507    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:36.398717 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:36.398639    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:36.398717 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:36.398708    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:38.398688272 +0000 UTC m=+39.086664349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:37.026255 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.026219    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lght6"]
Jul 14 21:51:37.051994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.051955    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lght6"]
Jul 14 21:51:37.052192 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.052039    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.054979 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.054951    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 21:51:37.055134 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.054990    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 21:51:37.056142 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.056060    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bh6h4\""
Jul 14 21:51:37.056142 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.056093    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 21:51:37.056142 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.056142    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 21:51:37.059963 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.059939    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:37.140882 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.140845    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:37.172978 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.172935    2570 generic.go:356] "Generic (PLEG): container finished" podID="23db57f2-1def-41e3-8e10-ed4c2297f176" containerID="fc2f9a839930702201442f52c35f6ca8145327e27dc482fe2469ed4d4a0ca6ec" exitCode=0
Jul 14 21:51:37.173453 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.173030    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerDied","Data":"fc2f9a839930702201442f52c35f6ca8145327e27dc482fe2469ed4d4a0ca6ec"}
Jul 14 21:51:37.205336 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.205294    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-crio-socket\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.205336 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.205336    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-data-volume\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.205565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.205424    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.205565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.205463    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.205565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.205503    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc69n\" (UniqueName: \"kubernetes.io/projected/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-api-access-mc69n\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.306984    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-crio-socket\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307040    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-data-volume\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307206    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-crio-socket\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307206    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.307288    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.307297    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.307384    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:37.807348805 +0000 UTC m=+38.495324891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : configmap "kube-rbac-proxy" not found
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.307404    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:37.807394602 +0000 UTC m=+38.495370678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:37.307487 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc69n\" (UniqueName: \"kubernetes.io/projected/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-api-access-mc69n\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.307879 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.307869    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-data-volume\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.318186 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.318147    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc69n\" (UniqueName: \"kubernetes.io/projected/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-api-access-mc69n\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.813196 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.813165    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.813347 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:37.813202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:37.813347 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.813284    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 21:51:37.813347 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.813302    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:37.813453 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.813352    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:38.813338267 +0000 UTC m=+39.501314339 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : configmap "kube-rbac-proxy" not found
Jul 14 21:51:37.813453 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:37.813370    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:38.813364239 +0000 UTC m=+39.501340309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:38.179637 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.179373    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" event={"ID":"23db57f2-1def-41e3-8e10-ed4c2297f176","Type":"ContainerStarted","Data":"5952cde5912e8dff26ad6f152fc0d4e37952a37a5fb1f064a5509f58e3ed6b67"}
Jul 14 21:51:38.180935 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.180909    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tlh5c" event={"ID":"322680e0-990d-495b-98d4-70629064e077","Type":"ContainerStarted","Data":"5b7e32be754944d6a1318860430fe4a10e92cb8f9f1f6b95a3c10bc37c8eb537"}
Jul 14 21:51:38.203250 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.203178    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gnrqt" podStartSLOduration=7.896348375 podStartE2EDuration="38.203153614s" totalImagesPullingTime="30.306805239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:38.200982619 +0000 UTC m=+38.888958741" watchObservedRunningTime="2026-07-14 21:51:38.203153614 +0000 UTC m=+38.891129708"
Jul 14 21:51:38.216402 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.216334    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tlh5c" podStartSLOduration=1.625305279 podStartE2EDuration="4.216316452s" totalImagesPullingTime="2.591011173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:38.215849127 +0000 UTC m=+38.903825222" watchObservedRunningTime="2026-07-14 21:51:38.216316452 +0000 UTC m=+38.904292584"
Jul 14 21:51:38.332445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.332401    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:38.402652 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.402625    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:38.420782 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.420736    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:38.420948 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.420910    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:38.420987 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.420974    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:42.420955673 +0000 UTC m=+43.108931744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:38.599761 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.599734    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnntz_7ccf0eb0-d809-494f-9a00-998e2c04af26/dns-node-resolver/0.log"
Jul 14 21:51:38.824440 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.824402    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:38.824440 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:38.824443    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:38.824655 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.824536    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 21:51:38.824655 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.824598    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:40.824582484 +0000 UTC m=+41.512558555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : configmap "kube-rbac-proxy" not found
Jul 14 21:51:38.824655 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.824612    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:38.824748 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:38.824658    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:40.824645595 +0000 UTC m=+41.512621666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:39.087535 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:39.087507    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:39.202765 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:39.202742    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z86g_19a004c4-cced-4841-9e55-bee9abe6d81c/node-ca/0.log"
Jul 14 21:51:39.783854 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:39.783824    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:39.887244 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:39.887202    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:40.200642 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:40.200620    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tlh5c_322680e0-990d-495b-98d4-70629064e077/serve-healthcheck-canary/0.log"
Jul 14 21:51:40.840978 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:40.840938    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:40.840978 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:40.840980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:40.841452 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:40.841084    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 21:51:40.841452 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:40.841101    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:40.841452 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:40.841158    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:44.841140805 +0000 UTC m=+45.529116875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : configmap "kube-rbac-proxy" not found
Jul 14 21:51:40.841452 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:40.841222    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:44.841205636 +0000 UTC m=+45.529181707 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:41.155614 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.155516    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:41.222246 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.222204    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:41.376927 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.376890    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"]
Jul 14 21:51:41.394609 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.394577    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"]
Jul 14 21:51:41.394744 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.394675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.397448 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.397419    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 14 21:51:41.397627 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.397456    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:41.398152 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.398119    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xh48x\""
Jul 14 21:51:41.398152 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.398119    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 14 21:51:41.398152 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.398120    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:41.471522 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.471491    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"]
Jul 14 21:51:41.489868 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.489838    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"]
Jul 14 21:51:41.490022 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.489912    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.492352 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.492321    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 14 21:51:41.492352 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.492327    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-j54vr\""
Jul 14 21:51:41.492565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.492366    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 14 21:51:41.492565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.492389    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:41.492762 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.492734    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:41.512029 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.511992    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-6sqmw"]
Jul 14 21:51:41.512197 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.512135    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.514687 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.514661    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 14 21:51:41.514687 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.514683    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6mbtm\""
Jul 14 21:51:41.514919 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.514718    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:41.514919 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.514669    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:41.534096 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.534057    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"]
Jul 14 21:51:41.534096 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.534097    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-6sqmw"]
Jul 14 21:51:41.534298 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.534115    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"]
Jul 14 21:51:41.534298 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.534216    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.536892 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.536869    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 14 21:51:41.537045 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.536958    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-spxpz\""
Jul 14 21:51:41.537119 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.537086    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 14 21:51:41.537629 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.537609    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:51:41.537729 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.537664    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 14 21:51:41.542864 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.542698    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 14 21:51:41.546712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.546046    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f9f61e4-41b3-4e1a-8542-39b4e269327a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.546712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.546439    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f9f61e4-41b3-4e1a-8542-39b4e269327a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.546712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.546594    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgj2h\" (UniqueName: \"kubernetes.io/projected/9f9f61e4-41b3-4e1a-8542-39b4e269327a-kube-api-access-tgj2h\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.647282 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647235    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f9f61e4-41b3-4e1a-8542-39b4e269327a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.647282 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647282    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/138a5340-83c5-4570-aeff-d12e703b5342-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.647482 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647418    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgj2h\" (UniqueName: \"kubernetes.io/projected/9f9f61e4-41b3-4e1a-8542-39b4e269327a-kube-api-access-tgj2h\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.647482 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-trusted-ca\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.647551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647490    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/138a5340-83c5-4570-aeff-d12e703b5342-config\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.647551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647508    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-config\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.647551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647522    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr6bv\" (UniqueName: \"kubernetes.io/projected/138a5340-83c5-4570-aeff-d12e703b5342-kube-api-access-xr6bv\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.647659 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647551    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f9f61e4-41b3-4e1a-8542-39b4e269327a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.647659 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlprl\" (UniqueName: \"kubernetes.io/projected/e515f36b-7029-42ae-b953-4220a93b734e-kube-api-access-wlprl\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.647659 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647647    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.647791 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e515f36b-7029-42ae-b953-4220a93b734e-serving-cert\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.647791 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647759    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jdrz\" (UniqueName: \"kubernetes.io/projected/583b6302-9404-47b3-a1ff-1d9cee78f3bc-kube-api-access-2jdrz\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.647908 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.647886    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f9f61e4-41b3-4e1a-8542-39b4e269327a-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.650532 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.650507    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f9f61e4-41b3-4e1a-8542-39b4e269327a-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.656194 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.656164    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgj2h\" (UniqueName: \"kubernetes.io/projected/9f9f61e4-41b3-4e1a-8542-39b4e269327a-kube-api-access-tgj2h\") pod \"kube-storage-version-migrator-operator-799cb7f958-vsttd\" (UID: \"9f9f61e4-41b3-4e1a-8542-39b4e269327a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.704500 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.704456    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"
Jul 14 21:51:41.748902 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.748860    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/138a5340-83c5-4570-aeff-d12e703b5342-config\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.748902 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.748901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-config\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.749148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.748920    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr6bv\" (UniqueName: \"kubernetes.io/projected/138a5340-83c5-4570-aeff-d12e703b5342-kube-api-access-xr6bv\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.749148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.748950    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlprl\" (UniqueName: \"kubernetes.io/projected/e515f36b-7029-42ae-b953-4220a93b734e-kube-api-access-wlprl\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.749148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749073    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.749148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749124    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e515f36b-7029-42ae-b953-4220a93b734e-serving-cert\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.749332 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jdrz\" (UniqueName: \"kubernetes.io/projected/583b6302-9404-47b3-a1ff-1d9cee78f3bc-kube-api-access-2jdrz\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.749332 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749219    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/138a5340-83c5-4570-aeff-d12e703b5342-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.749332 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-trusted-ca\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.749900 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749543    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/138a5340-83c5-4570-aeff-d12e703b5342-config\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.749900 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:41.749589    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 21:51:41.749900 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.749646    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-config\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.749900 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:41.749661    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls podName:583b6302-9404-47b3-a1ff-1d9cee78f3bc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:42.249639787 +0000 UTC m=+42.937615858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls") pod "cluster-samples-operator-65db858894-gf242" (UID: "583b6302-9404-47b3-a1ff-1d9cee78f3bc") : secret "samples-operator-tls" not found
Jul 14 21:51:41.750233 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.750210    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e515f36b-7029-42ae-b953-4220a93b734e-trusted-ca\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.751579 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.751550    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/138a5340-83c5-4570-aeff-d12e703b5342-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.751708 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.751687    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e515f36b-7029-42ae-b953-4220a93b734e-serving-cert\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.758086 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.758040    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr6bv\" (UniqueName: \"kubernetes.io/projected/138a5340-83c5-4570-aeff-d12e703b5342-kube-api-access-xr6bv\") pod \"service-ca-operator-7695fc6c8c-8h44j\" (UID: \"138a5340-83c5-4570-aeff-d12e703b5342\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.758230 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.758134    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlprl\" (UniqueName: \"kubernetes.io/projected/e515f36b-7029-42ae-b953-4220a93b734e-kube-api-access-wlprl\") pod \"console-operator-7866497494-6sqmw\" (UID: \"e515f36b-7029-42ae-b953-4220a93b734e\") " pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.758230 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.758199    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jdrz\" (UniqueName: \"kubernetes.io/projected/583b6302-9404-47b3-a1ff-1d9cee78f3bc-kube-api-access-2jdrz\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:41.801275 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.801239    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"
Jul 14 21:51:41.845457 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.845416    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:41.851959 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.851925    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd"]
Jul 14 21:51:41.855835 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:41.855774    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9f61e4_41b3_4e1a_8542_39b4e269327a.slice/crio-78b02efee421b895ba47e88943ffb663c27782231a3e9d73bdad2e750ed4d71a WatchSource:0}: Error finding container 78b02efee421b895ba47e88943ffb663c27782231a3e9d73bdad2e750ed4d71a: Status 404 returned error can't find the container with id 78b02efee421b895ba47e88943ffb663c27782231a3e9d73bdad2e750ed4d71a
Jul 14 21:51:41.977383 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:41.977282    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j"]
Jul 14 21:51:41.980587 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:41.980556    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138a5340_83c5_4570_aeff_d12e703b5342.slice/crio-0e7e587538b815c5a6419ed3af00c1f954d42e212b69f6cd934134004f10e543 WatchSource:0}: Error finding container 0e7e587538b815c5a6419ed3af00c1f954d42e212b69f6cd934134004f10e543: Status 404 returned error can't find the container with id 0e7e587538b815c5a6419ed3af00c1f954d42e212b69f6cd934134004f10e543
Jul 14 21:51:42.020421 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.020381    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-6sqmw"]
Jul 14 21:51:42.025080 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:42.025046    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode515f36b_7029_42ae_b953_4220a93b734e.slice/crio-aa0730bbcd738f645106a6c3cd57063b0d482711a01d4c609a6e3124ff866e41 WatchSource:0}: Error finding container aa0730bbcd738f645106a6c3cd57063b0d482711a01d4c609a6e3124ff866e41: Status 404 returned error can't find the container with id aa0730bbcd738f645106a6c3cd57063b0d482711a01d4c609a6e3124ff866e41
Jul 14 21:51:42.191416 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.191383    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6sqmw" event={"ID":"e515f36b-7029-42ae-b953-4220a93b734e","Type":"ContainerStarted","Data":"aa0730bbcd738f645106a6c3cd57063b0d482711a01d4c609a6e3124ff866e41"}
Jul 14 21:51:42.192477 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.192442    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j" event={"ID":"138a5340-83c5-4570-aeff-d12e703b5342","Type":"ContainerStarted","Data":"0e7e587538b815c5a6419ed3af00c1f954d42e212b69f6cd934134004f10e543"}
Jul 14 21:51:42.193444 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.193420    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd" event={"ID":"9f9f61e4-41b3-4e1a-8542-39b4e269327a","Type":"ContainerStarted","Data":"78b02efee421b895ba47e88943ffb663c27782231a3e9d73bdad2e750ed4d71a"}
Jul 14 21:51:42.253997 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.253901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:42.254160 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:42.254030    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 21:51:42.254160 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:42.254085    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls podName:583b6302-9404-47b3-a1ff-1d9cee78f3bc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:43.25407068 +0000 UTC m=+43.942046751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls") pod "cluster-samples-operator-65db858894-gf242" (UID: "583b6302-9404-47b3-a1ff-1d9cee78f3bc") : secret "samples-operator-tls" not found
Jul 14 21:51:42.455183 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.455145    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:42.455340 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:42.455279    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:42.455394 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:42.455360    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:50.455327915 +0000 UTC m=+51.143304004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:42.930242 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:42.930114    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:43.030783 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:43.030740    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:43.263024 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:43.262989    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:43.263198 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:43.263156    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 21:51:43.263240 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:43.263222    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls podName:583b6302-9404-47b3-a1ff-1d9cee78f3bc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:45.263206514 +0000 UTC m=+45.951182586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls") pod "cluster-samples-operator-65db858894-gf242" (UID: "583b6302-9404-47b3-a1ff-1d9cee78f3bc") : secret "samples-operator-tls" not found
Jul 14 21:51:44.274430 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:44.274366    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:44.334237 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:44.334206    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:44.878676 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:44.878636    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:44.878676 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:44.878679    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:44.878964 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:44.878782    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 21:51:44.878964 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:44.878877    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:44.878964 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:44.878881    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:52.878861685 +0000 UTC m=+53.566837771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : configmap "kube-rbac-proxy" not found
Jul 14 21:51:44.878964 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:44.878936    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:51:52.878920314 +0000 UTC m=+53.566896385 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:45.282739 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:45.282692    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:45.283223 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:45.282899    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 21:51:45.283223 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:45.282982    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls podName:583b6302-9404-47b3-a1ff-1d9cee78f3bc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:49.282960741 +0000 UTC m=+49.970936813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls") pod "cluster-samples-operator-65db858894-gf242" (UID: "583b6302-9404-47b3-a1ff-1d9cee78f3bc") : secret "samples-operator-tls" not found
Jul 14 21:51:45.304786 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:45.304735    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:45.405359 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:45.405321    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:45.768233 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:45.768196    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:45.827417 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:45.827386    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:46.206399 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.206371    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/0.log"
Jul 14 21:51:46.206617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.206417    2570 generic.go:356] "Generic (PLEG): container finished" podID="e515f36b-7029-42ae-b953-4220a93b734e" containerID="463390cad15145b49c83627671fc1dea3110116f217f8f4a7e152116da98c852" exitCode=255
Jul 14 21:51:46.206617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.206510    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6sqmw" event={"ID":"e515f36b-7029-42ae-b953-4220a93b734e","Type":"ContainerDied","Data":"463390cad15145b49c83627671fc1dea3110116f217f8f4a7e152116da98c852"}
Jul 14 21:51:46.206761 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.206739    2570 scope.go:117] "RemoveContainer" containerID="463390cad15145b49c83627671fc1dea3110116f217f8f4a7e152116da98c852"
Jul 14 21:51:46.208282 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.208258    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j" event={"ID":"138a5340-83c5-4570-aeff-d12e703b5342","Type":"ContainerStarted","Data":"cfde6948fd12e323c782e699622dd747a9f23f41ddf56b0749c74f4b125b465d"}
Jul 14 21:51:46.209918 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.209894    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd" event={"ID":"9f9f61e4-41b3-4e1a-8542-39b4e269327a","Type":"ContainerStarted","Data":"d0350812a5e03ecafdb918fa835dbb418820936acf9f7035c910eb59d025246a"}
Jul 14 21:51:46.239605 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:46.239545    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j" podStartSLOduration=1.351517078 podStartE2EDuration="5.239522137s" totalImagesPullingTime="3.888005059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:46.238555155 +0000 UTC m=+46.926531249" watchObservedRunningTime="2026-07-14 21:51:46.239522137 +0000 UTC m=+46.927498223"
Jul 14 21:51:47.215036 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215008    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 21:51:47.215493 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215467    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/0.log"
Jul 14 21:51:47.215640 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215509    2570 generic.go:356] "Generic (PLEG): container finished" podID="e515f36b-7029-42ae-b953-4220a93b734e" containerID="3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd" exitCode=255
Jul 14 21:51:47.215640 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215543    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6sqmw" event={"ID":"e515f36b-7029-42ae-b953-4220a93b734e","Type":"ContainerDied","Data":"3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd"}
Jul 14 21:51:47.215640 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215630    2570 scope.go:117] "RemoveContainer" containerID="463390cad15145b49c83627671fc1dea3110116f217f8f4a7e152116da98c852"
Jul 14 21:51:47.215921 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.215899    2570 scope.go:117] "RemoveContainer" containerID="3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd"
Jul 14 21:51:47.216251 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:47.216098    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6sqmw_openshift-console-operator(e515f36b-7029-42ae-b953-4220a93b734e)\"" pod="openshift-console-operator/console-operator-7866497494-6sqmw" podUID="e515f36b-7029-42ae-b953-4220a93b734e"
Jul 14 21:51:47.234449 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.234394    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd" podStartSLOduration=2.226162781 podStartE2EDuration="6.234373815s" totalImagesPullingTime="4.008211034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:46.254640844 +0000 UTC m=+46.942616936" watchObservedRunningTime="2026-07-14 21:51:47.234373815 +0000 UTC m=+47.922349935"
Jul 14 21:51:47.422135 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.422093    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:47.516702 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.516582    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:47.620554 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.620509    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:47.718511 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.718474    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:47.819437 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.819349    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:47.977465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:47.977428    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:48.035089 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:48.035057    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:48.220287 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:48.220259    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 21:51:48.220725 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:48.220615    2570 scope.go:117] "RemoveContainer" containerID="3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd"
Jul 14 21:51:48.220797 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:48.220765    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6sqmw_openshift-console-operator(e515f36b-7029-42ae-b953-4220a93b734e)\"" pod="openshift-console-operator/console-operator-7866497494-6sqmw" podUID="e515f36b-7029-42ae-b953-4220a93b734e"
Jul 14 21:51:48.421228 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:48.421190    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:48.521754 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:48.521660    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:49.321572 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:49.321534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:49.322001 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:49.321714    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 14 21:51:49.322001 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:49.321793    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls podName:583b6302-9404-47b3-a1ff-1d9cee78f3bc nodeName:}" failed. No retries permitted until 2026-07-14 21:51:57.321775595 +0000 UTC m=+58.009751679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls") pod "cluster-samples-operator-65db858894-gf242" (UID: "583b6302-9404-47b3-a1ff-1d9cee78f3bc") : secret "samples-operator-tls" not found
Jul 14 21:51:50.021113 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.021075    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-pqwnk"]
Jul 14 21:51:50.030650 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.030616    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.033387 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.033354    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-pqwnk"]
Jul 14 21:51:50.033729 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.033706    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 14 21:51:50.033880 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.033786    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 14 21:51:50.034521 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.034496    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-bzl85\""
Jul 14 21:51:50.034633 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.034499    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 14 21:51:50.034633 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.034588    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 14 21:51:50.058095 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.058063    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:50.122873 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.122819    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:50.228866 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.228831    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5265e768-52b9-461c-9e99-43f06426985f-signing-key\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.229091 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.228905    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xw4\" (UniqueName: \"kubernetes.io/projected/5265e768-52b9-461c-9e99-43f06426985f-kube-api-access-z5xw4\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.229091 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.228939    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5265e768-52b9-461c-9e99-43f06426985f-signing-cabundle\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.255673 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.255635    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:50.315589 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.315506    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:50.329705 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.329664    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5265e768-52b9-461c-9e99-43f06426985f-signing-key\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.330213 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.329730    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xw4\" (UniqueName: \"kubernetes.io/projected/5265e768-52b9-461c-9e99-43f06426985f-kube-api-access-z5xw4\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.330213 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.329873    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5265e768-52b9-461c-9e99-43f06426985f-signing-cabundle\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.330445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.330428    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5265e768-52b9-461c-9e99-43f06426985f-signing-cabundle\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.332191 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.332170    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5265e768-52b9-461c-9e99-43f06426985f-signing-key\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.339204 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.339181    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xw4\" (UniqueName: \"kubernetes.io/projected/5265e768-52b9-461c-9e99-43f06426985f-kube-api-access-z5xw4\") pod \"service-ca-79f68b9f4f-pqwnk\" (UID: \"5265e768-52b9-461c-9e99-43f06426985f\") " pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.340986 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.340967    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk"
Jul 14 21:51:50.487133 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.487101    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-pqwnk"]
Jul 14 21:51:50.489927 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:51:50.489899    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5265e768_52b9_461c_9e99_43f06426985f.slice/crio-d82bb66e3080c8ce239fc0bb7313b0cdbb5ef05c410a38f52fd7032665bbffdf WatchSource:0}: Error finding container d82bb66e3080c8ce239fc0bb7313b0cdbb5ef05c410a38f52fd7032665bbffdf: Status 404 returned error can't find the container with id d82bb66e3080c8ce239fc0bb7313b0cdbb5ef05c410a38f52fd7032665bbffdf
Jul 14 21:51:50.532090 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:50.532068    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:51:50.532201 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:50.532187    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 21:51:50.532252 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:50.532243    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls podName:f2ff2305-b9cc-46d7-b766-2d3697a1e7cc nodeName:}" failed. No retries permitted until 2026-07-14 21:52:06.532228779 +0000 UTC m=+67.220204849 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls") pod "dns-default-qgrtx" (UID: "f2ff2305-b9cc-46d7-b766-2d3697a1e7cc") : secret "dns-default-metrics-tls" not found
Jul 14 21:51:51.233874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.233838    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk" event={"ID":"5265e768-52b9-461c-9e99-43f06426985f","Type":"ContainerStarted","Data":"8679f339e86b9e38cb5b8d965c04f637211afd6b20380f9a047736bea4c72546"}
Jul 14 21:51:51.234028 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.233920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk" event={"ID":"5265e768-52b9-461c-9e99-43f06426985f","Type":"ContainerStarted","Data":"d82bb66e3080c8ce239fc0bb7313b0cdbb5ef05c410a38f52fd7032665bbffdf"}
Jul 14 21:51:51.236708 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.236674    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:51.845904 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.845863    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:51.846372 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.845935    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:51:51.846442 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:51.846384    2570 scope.go:117] "RemoveContainer" containerID="3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd"
Jul 14 21:51:51.846590 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:51.846565    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-6sqmw_openshift-console-operator(e515f36b-7029-42ae-b953-4220a93b734e)\"" pod="openshift-console-operator/console-operator-7866497494-6sqmw" podUID="e515f36b-7029-42ae-b953-4220a93b734e"
Jul 14 21:51:52.011856 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.011821    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:52.266457 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.266425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:52.510932 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.510878    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:52.615200 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.615105    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:52.726593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.726559    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:52.948952 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.948918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:52.948952 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.948953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:52.949466 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:52.949063    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:52.949466 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:51:52.949117    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls podName:8d788a96-bf9b-4958-b4c5-d0ef6893b7cf nodeName:}" failed. No retries permitted until 2026-07-14 21:52:08.949104268 +0000 UTC m=+69.637080338 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lght6" (UID: "8d788a96-bf9b-4958-b4c5-d0ef6893b7cf") : secret "insights-runtime-extractor-tls" not found
Jul 14 21:51:52.949466 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:52.949436    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:51:53.512814 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:53.512777    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:53.912535 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:53.912440    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:54.111930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:54.111899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:54.286409 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:54.286370    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:54.352418 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:54.352380    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:54.510837 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:54.510784    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:54.922421 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:54.922382    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:55.022573 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:55.022534    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:55.671930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:55.671900    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:55.770241 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:55.770207    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:55.916704 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:55.916666    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:56.304660 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:56.304622    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:56.504453 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:56.504413    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:56.605153 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:56.605061    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:56.906530 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:56.906441    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:57.387322 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.387285    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:57.389669 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.389641    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/583b6302-9404-47b3-a1ff-1d9cee78f3bc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-gf242\" (UID: \"583b6302-9404-47b3-a1ff-1d9cee78f3bc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:57.421785 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.421744    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"
Jul 14 21:51:57.581881 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.581828    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-pqwnk" podStartSLOduration=7.581793084 podStartE2EDuration="7.581793084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:51:51.251779942 +0000 UTC m=+51.939756036" watchObservedRunningTime="2026-07-14 21:51:57.581793084 +0000 UTC m=+58.269769176"
Jul 14 21:51:57.582079 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.581965    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242"]
Jul 14 21:51:57.706193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:57.706152    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:58.110926 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.110891    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:58.258193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.258149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242" event={"ID":"583b6302-9404-47b3-a1ff-1d9cee78f3bc","Type":"ContainerStarted","Data":"500eb2ba452b81c21fdfb1df820135c97c39c7f283fc5be73e8ac77221ca3b94"}
Jul 14 21:51:58.299494 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.299447    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:58.362280 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.362187    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:58.514785 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.514751    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:58.709797 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:58.709761    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:51:59.922977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:51:59.922879    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:00.111568 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.111534    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:00.267582 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.267544    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242" event={"ID":"583b6302-9404-47b3-a1ff-1d9cee78f3bc","Type":"ContainerStarted","Data":"97c4266e23e570b2907cc52e424a568157274884ab5a4ec6575219983dc84515"}
Jul 14 21:52:00.267783 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.267601    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242" event={"ID":"583b6302-9404-47b3-a1ff-1d9cee78f3bc","Type":"ContainerStarted","Data":"186a74a38e747d19e1fab4a9b6f8932e8e92d52b030c92138ba9aa72d170f4a8"}
Jul 14 21:52:00.283682 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.283629    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-gf242" podStartSLOduration=17.49386689 podStartE2EDuration="19.283613164s" totalImagesPullingTime="1.789746274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:00.283062931 +0000 UTC m=+60.971039025" watchObservedRunningTime="2026-07-14 21:52:00.283613164 +0000 UTC m=+60.971589300"
Jul 14 21:52:00.305056 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.305020    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:00.508959 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.508925    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:00.733138 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:00.733103    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:01.159919 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:01.159843    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdvlx"
Jul 14 21:52:01.518436 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:01.518403    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:02.115721 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:02.115680    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:02.311021 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:02.310989    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:02.373728 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:02.373645    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:02.513107 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:02.513065    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:02.711372 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:02.711329    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:03.870427 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:03.870388    2570 scope.go:117] "RemoveContainer" containerID="3ecf3fa7f1bf99c0df74cb956b18e142ec049ae9eb875e020262c3374467c2cd"
Jul 14 21:52:04.282768 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.282735    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 21:52:04.282951 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.282839    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-6sqmw" event={"ID":"e515f36b-7029-42ae-b953-4220a93b734e","Type":"ContainerStarted","Data":"ca40ea63d3dd70aede781fd789b9f5b93187debd25c32650c7297d61105d84f9"}
Jul 14 21:52:04.283147 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.283117    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:52:04.304242 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.304182    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-6sqmw" podStartSLOduration=19.464968673 podStartE2EDuration="23.30416077s" totalImagesPullingTime="3.839192097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:04.303653575 +0000 UTC m=+64.991629669" watchObservedRunningTime="2026-07-14 21:52:04.30416077 +0000 UTC m=+64.992136842"
Jul 14 21:52:04.417899 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.417870    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-6sqmw"
Jul 14 21:52:04.602036 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.601942    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:04.695996 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.695956    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:04.892478 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:04.892379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.278380 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.278346    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.337857 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.337825    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.662593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.662483    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:52:05.662593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.662530    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:52:05.662593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.662588    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:52:05.665439 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.665413    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 21:52:05.665564 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.665473    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 21:52:05.666389 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.666372    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 21:52:05.675676 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.675647    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdc7e983-6358-4474-9304-02e33c48c692-metrics-certs\") pod \"network-metrics-daemon-t9h8k\" (UID: \"fdc7e983-6358-4474-9304-02e33c48c692\") " pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:52:05.675676 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.675665    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/385bbb00-e382-4a0b-927b-13a51ee2d1d3-original-pull-secret\") pod \"global-pull-secret-syncer-55sxh\" (UID: \"385bbb00-e382-4a0b-927b-13a51ee2d1d3\") " pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:52:05.675944 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.675929    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 21:52:05.686656 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.686624    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qfhw\" (UniqueName: \"kubernetes.io/projected/dcb2eedd-b418-4246-9dac-b4175d983303-kube-api-access-5qfhw\") pod \"network-check-target-mk8w6\" (UID: \"dcb2eedd-b418-4246-9dac-b4175d983303\") " pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:52:05.687552 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.687499    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.783778 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.783738    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l5sfp\""
Jul 14 21:52:05.789070 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.789044    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.791796 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.791779    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t9h8k"
Jul 14 21:52:05.796096 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.796071    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-r4xkn\""
Jul 14 21:52:05.804513 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.804482    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:52:05.806654 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.806617    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-55sxh"
Jul 14 21:52:05.906247 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.906201    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:05.948398 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:05.948362    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t9h8k"]
Jul 14 21:52:05.951274 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:05.951240    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc7e983_6358_4474_9304_02e33c48c692.slice/crio-7a566564d46be9002e72969ce083cb60a1e16eaac3f1911fa90f206a2c0137da WatchSource:0}: Error finding container 7a566564d46be9002e72969ce083cb60a1e16eaac3f1911fa90f206a2c0137da: Status 404 returned error can't find the container with id 7a566564d46be9002e72969ce083cb60a1e16eaac3f1911fa90f206a2c0137da
Jul 14 21:52:06.184972 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.184923    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mk8w6"]
Jul 14 21:52:06.187036 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.186905    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-55sxh"]
Jul 14 21:52:06.188953 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:06.188927    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcb2eedd_b418_4246_9dac_b4175d983303.slice/crio-634a615f91fb75495d3b7a39b21df7610d5a07ecbc6e3ebea69dee26ffa69cb1 WatchSource:0}: Error finding container 634a615f91fb75495d3b7a39b21df7610d5a07ecbc6e3ebea69dee26ffa69cb1: Status 404 returned error can't find the container with id 634a615f91fb75495d3b7a39b21df7610d5a07ecbc6e3ebea69dee26ffa69cb1
Jul 14 21:52:06.190692 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:06.190627    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod385bbb00_e382_4a0b_927b_13a51ee2d1d3.slice/crio-65f52a58ab2d8acc2d9676fb7f45a5edd69872b1a4800ba725c685e7949f193a WatchSource:0}: Error finding container 65f52a58ab2d8acc2d9676fb7f45a5edd69872b1a4800ba725c685e7949f193a: Status 404 returned error can't find the container with id 65f52a58ab2d8acc2d9676fb7f45a5edd69872b1a4800ba725c685e7949f193a
Jul 14 21:52:06.216034 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.215939    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:06.290901 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.290858    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mk8w6" event={"ID":"dcb2eedd-b418-4246-9dac-b4175d983303","Type":"ContainerStarted","Data":"634a615f91fb75495d3b7a39b21df7610d5a07ecbc6e3ebea69dee26ffa69cb1"}
Jul 14 21:52:06.292145 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.292112    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-55sxh" event={"ID":"385bbb00-e382-4a0b-927b-13a51ee2d1d3","Type":"ContainerStarted","Data":"65f52a58ab2d8acc2d9676fb7f45a5edd69872b1a4800ba725c685e7949f193a"}
Jul 14 21:52:06.293257 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.293235    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9h8k" event={"ID":"fdc7e983-6358-4474-9304-02e33c48c692","Type":"ContainerStarted","Data":"7a566564d46be9002e72969ce083cb60a1e16eaac3f1911fa90f206a2c0137da"}
Jul 14 21:52:06.340440 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.340401    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:06.497895 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.497789    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:06.571990 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.571952    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:52:06.574631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.574598    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2ff2305-b9cc-46d7-b766-2d3697a1e7cc-metrics-tls\") pod \"dns-default-qgrtx\" (UID: \"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc\") " pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:52:06.715079 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.715005    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:06.793316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.793024    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2wg9m\""
Jul 14 21:52:06.800875 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:06.800089    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:52:07.044132 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.044029    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:07.058578 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.058544    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qgrtx"]
Jul 14 21:52:07.063418 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:07.063381    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ff2305_b9cc_46d7_b766_2d3697a1e7cc.slice/crio-93c0ffcb3a520a706867963703f1c0a440d31860c1b0f241975b48eb952d473d WatchSource:0}: Error finding container 93c0ffcb3a520a706867963703f1c0a440d31860c1b0f241975b48eb952d473d: Status 404 returned error can't find the container with id 93c0ffcb3a520a706867963703f1c0a440d31860c1b0f241975b48eb952d473d
Jul 14 21:52:07.301672 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.301432    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9h8k" event={"ID":"fdc7e983-6358-4474-9304-02e33c48c692","Type":"ContainerStarted","Data":"179691255ebe2df06f72564fce9c930d2b660328e437d64d437fcb66506818f1"}
Jul 14 21:52:07.301672 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.301499    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t9h8k" event={"ID":"fdc7e983-6358-4474-9304-02e33c48c692","Type":"ContainerStarted","Data":"4d97780eb273160dbd3ee590623d81dbb517f97919784b8fa53a9065dbcaddc4"}
Jul 14 21:52:07.303131 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.303096    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgrtx" event={"ID":"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc","Type":"ContainerStarted","Data":"93c0ffcb3a520a706867963703f1c0a440d31860c1b0f241975b48eb952d473d"}
Jul 14 21:52:07.318785 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.318632    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t9h8k" podStartSLOduration=67.352433686 podStartE2EDuration="1m8.318608246s" totalImagesPullingTime="966.17456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:07.31815674 +0000 UTC m=+68.006132833" watchObservedRunningTime="2026-07-14 21:52:07.318608246 +0000 UTC m=+68.006584340"
Jul 14 21:52:07.512445 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.512408    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:07.826492 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:07.826458    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:08.992816 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:08.992745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:52:08.996031 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:08.995958    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8d788a96-bf9b-4958-b4c5-d0ef6893b7cf-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lght6\" (UID: \"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf\") " pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:52:09.164050 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:09.164016    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bh6h4\""
Jul 14 21:52:09.172112 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:09.172079    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lght6"
Jul 14 21:52:09.275643 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:09.275540    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:09.616538 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:09.616436    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:11.886326 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:11.886037    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lght6"]
Jul 14 21:52:12.329162 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.328405    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-55sxh" event={"ID":"385bbb00-e382-4a0b-927b-13a51ee2d1d3","Type":"ContainerStarted","Data":"15f5f91efeb1e003fb3e6a0aa49d8ac8bcaf7c95c22ae1084ea8f9ccf7d7f225"}
Jul 14 21:52:12.333507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.333360    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgrtx" event={"ID":"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc","Type":"ContainerStarted","Data":"e95f9d4e320b4c2122642b3c927ea36955546509ce1c1aa5aed609c09733bdec"}
Jul 14 21:52:12.333507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.333422    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:52:12.333507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.333450    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qgrtx" event={"ID":"f2ff2305-b9cc-46d7-b766-2d3697a1e7cc","Type":"ContainerStarted","Data":"1d77539d900a5aa124423a18fb8294ac8bdbd0b3ee4fb9bc6c56163050074232"}
Jul 14 21:52:12.335496 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.335407    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lght6" event={"ID":"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf","Type":"ContainerStarted","Data":"54bc0d6e1c4bc5d46533a7c613d60cb17ab9d928934752534063bb088123624f"}
Jul 14 21:52:12.338002 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.337929    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mk8w6" event={"ID":"dcb2eedd-b418-4246-9dac-b4175d983303","Type":"ContainerStarted","Data":"9fd7d888df6a5759d8523b3bb8e5a03dd50e56875bf8a17cef24212f65494d5a"}
Jul 14 21:52:12.338267 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.338229    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:52:12.344603 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.344532    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-55sxh" podStartSLOduration=66.796883394 podStartE2EDuration="1m12.344507634s" totalImagesPullingTime="5.54762424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:12.343603326 +0000 UTC m=+73.031579461" watchObservedRunningTime="2026-07-14 21:52:12.344507634 +0000 UTC m=+73.032483730"
Jul 14 21:52:12.363679 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.363305    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mk8w6" podStartSLOduration=67.830598101 podStartE2EDuration="1m13.363287277s" totalImagesPullingTime="5.532689176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:12.362678573 +0000 UTC m=+73.050654667" watchObservedRunningTime="2026-07-14 21:52:12.363287277 +0000 UTC m=+73.051263370"
Jul 14 21:52:12.385586 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:12.384200    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qgrtx" podStartSLOduration=33.728974641 podStartE2EDuration="38.38416492s" totalImagesPullingTime="4.655190279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:12.378135736 +0000 UTC m=+73.066111830" watchObservedRunningTime="2026-07-14 21:52:12.38416492 +0000 UTC m=+73.072141020"
Jul 14 21:52:14.756487 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.756451    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"]
Jul 14 21:52:14.762507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.762475    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.765192 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.764960    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 14 21:52:14.766112 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766088    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 14 21:52:14.766112 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766105    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 14 21:52:14.766327 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766306    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 14 21:52:14.766401 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766329    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 14 21:52:14.766456 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766433    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 14 21:52:14.766533 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.766501    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 14 21:52:14.773649 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.773625    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"]
Jul 14 21:52:14.838489 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838450    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.838677 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838502    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99mnn\" (UniqueName: \"kubernetes.io/projected/04b6bd13-f265-4e9d-b47c-cd648492334b-kube-api-access-99mnn\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.838677 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838563    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b6bd13-f265-4e9d-b47c-cd648492334b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.838677 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.838677 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838614    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.838677 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.838642    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.892050 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.892018    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75f9c687fc-2cf7f"]
Jul 14 21:52:14.897696 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.897662    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:14.900845 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.900817    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 21:52:14.901012 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.900868    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 21:52:14.901435 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.901413    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dhwt6\""
Jul 14 21:52:14.901435 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.901428    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 21:52:14.930007 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.928738    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 21:52:14.930007 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.929309    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f9c687fc-2cf7f"]
Jul 14 21:52:14.934296 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.934224    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:14.939594 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.939563    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.939903 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.939882    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99mnn\" (UniqueName: \"kubernetes.io/projected/04b6bd13-f265-4e9d-b47c-cd648492334b-kube-api-access-99mnn\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.940098 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.940071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b6bd13-f265-4e9d-b47c-cd648492334b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.940235 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.940105    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.940235 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.940133    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.940235 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.940164    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.940906 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.940882    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b6bd13-f265-4e9d-b47c-cd648492334b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.943519 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.943492    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.943519 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.943515    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-ca\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.943784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.943621    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-hub\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.943784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.943638    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/04b6bd13-f265-4e9d-b47c-cd648492334b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:14.954939 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:14.954877    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99mnn\" (UniqueName: \"kubernetes.io/projected/04b6bd13-f265-4e9d-b47c-cd648492334b-kube-api-access-99mnn\") pod \"cluster-proxy-proxy-agent-76cff7b8c7-lxgnq\" (UID: \"04b6bd13-f265-4e9d-b47c-cd648492334b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:15.023401 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.023318    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:15.041146 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041108    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwss4\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-kube-api-access-jwss4\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-image-registry-private-configuration\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041204    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-installation-pull-secrets\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041227    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-bound-sa-token\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041284    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-tls\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-ca-trust-extracted\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041544 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041329    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-trusted-ca\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.041544 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.041349    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-certificates\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.084011 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.083975    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"
Jul 14 21:52:15.126265 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.126218    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:15.141826 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141765    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-image-registry-private-configuration\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.141826 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141828    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-installation-pull-secrets\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141856    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-bound-sa-token\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141879    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-tls\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-ca-trust-extracted\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141931    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-trusted-ca\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.141957    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-certificates\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.142013    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwss4\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-kube-api-access-jwss4\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142502 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.142460    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-ca-trust-extracted\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.142992 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.142963    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-certificates\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.143107 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.143054    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-trusted-ca\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.144946 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.144922    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-registry-tls\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.145333 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.145306    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-image-registry-private-configuration\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.145484 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.145396    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-installation-pull-secrets\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.156641 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.156611    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-bound-sa-token\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.164376 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.164349    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwss4\" (UniqueName: \"kubernetes.io/projected/fdcf2fe2-f57d-4382-9dcf-bd603ec06030-kube-api-access-jwss4\") pod \"image-registry-75f9c687fc-2cf7f\" (UID: \"fdcf2fe2-f57d-4382-9dcf-bd603ec06030\") " pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.233900 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.233860    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:15.686836 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.685928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq"]
Jul 14 21:52:15.691314 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:15.691287    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04b6bd13_f265_4e9d_b47c_cd648492334b.slice/crio-e24ad2c37174f2e0a8b86e56ff583ab1fd0e2fb82773c6f18489866e6aaa4691 WatchSource:0}: Error finding container e24ad2c37174f2e0a8b86e56ff583ab1fd0e2fb82773c6f18489866e6aaa4691: Status 404 returned error can't find the container with id e24ad2c37174f2e0a8b86e56ff583ab1fd0e2fb82773c6f18489866e6aaa4691
Jul 14 21:52:15.702019 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:15.701988    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f9c687fc-2cf7f"]
Jul 14 21:52:15.705813 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:15.705771    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcf2fe2_f57d_4382_9dcf_bd603ec06030.slice/crio-63f73cf9a5314dc4c134e02ac0c901d78aa1bf20d6324c9a849196dcf7283248 WatchSource:0}: Error finding container 63f73cf9a5314dc4c134e02ac0c901d78aa1bf20d6324c9a849196dcf7283248: Status 404 returned error can't find the container with id 63f73cf9a5314dc4c134e02ac0c901d78aa1bf20d6324c9a849196dcf7283248
Jul 14 21:52:16.355850 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.355814    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lght6" event={"ID":"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf","Type":"ContainerStarted","Data":"98811f8877d6b1ee43b30191cef66affb2e0c193e98396d9af61f462453d0ea7"}
Jul 14 21:52:16.357304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.357265    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f" event={"ID":"fdcf2fe2-f57d-4382-9dcf-bd603ec06030","Type":"ContainerStarted","Data":"8520831375afcba86e524e4f24ed08dc0aab30097cf9e8d27ff714f2c91e2e0a"}
Jul 14 21:52:16.357442 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.357322    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f" event={"ID":"fdcf2fe2-f57d-4382-9dcf-bd603ec06030","Type":"ContainerStarted","Data":"63f73cf9a5314dc4c134e02ac0c901d78aa1bf20d6324c9a849196dcf7283248"}
Jul 14 21:52:16.357500 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.357451    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:16.358357 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.358334    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq" event={"ID":"04b6bd13-f265-4e9d-b47c-cd648492334b","Type":"ContainerStarted","Data":"e24ad2c37174f2e0a8b86e56ff583ab1fd0e2fb82773c6f18489866e6aaa4691"}
Jul 14 21:52:16.376386 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:16.376334    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f" podStartSLOduration=2.37630926 podStartE2EDuration="2.37630926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:16.375894726 +0000 UTC m=+77.063870845" watchObservedRunningTime="2026-07-14 21:52:16.37630926 +0000 UTC m=+77.064285357"
Jul 14 21:52:17.369437 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:17.369377    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lght6" event={"ID":"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf","Type":"ContainerStarted","Data":"63d7db60ad6ede890c71f1ecfd5a49f3802867c28f277e334d527e66acba6640"}
Jul 14 21:52:20.514264 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:20.514230    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:20.617172 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:20.617137    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:20.715049 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:20.714956    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:21.385698 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.385651    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq" event={"ID":"04b6bd13-f265-4e9d-b47c-cd648492334b","Type":"ContainerStarted","Data":"ea970cfa4afa5ac02599e26cdd3b7ec18a76863168b7b797bddf169fb98302b1"}
Jul 14 21:52:21.385922 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.385706    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq" event={"ID":"04b6bd13-f265-4e9d-b47c-cd648492334b","Type":"ContainerStarted","Data":"d1c8a8bb5634f31413d616584128ee5c4248bd4ec05eebd0f99c602aba5b83f1"}
Jul 14 21:52:21.385922 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.385723    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq" event={"ID":"04b6bd13-f265-4e9d-b47c-cd648492334b","Type":"ContainerStarted","Data":"9ede7d6dcdcee80107faff1efffb75b19063b157c9e43b55afc5717c82ef38c8"}
Jul 14 21:52:21.387842 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.387794    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lght6" event={"ID":"8d788a96-bf9b-4958-b4c5-d0ef6893b7cf","Type":"ContainerStarted","Data":"5b71ca520823c1f1a11b0da65a102dbf051d8695fe6792615f9c5b6b8bd4d5d9"}
Jul 14 21:52:21.407152 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.407096    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-76cff7b8c7-lxgnq" podStartSLOduration=2.630363058 podStartE2EDuration="7.407078784s" totalImagesPullingTime="4.776715726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:52:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:21.40548961 +0000 UTC m=+82.093465732" watchObservedRunningTime="2026-07-14 21:52:21.407078784 +0000 UTC m=+82.095054877"
Jul 14 21:52:21.422471 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:21.422417    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lght6" podStartSLOduration=36.991643818 podStartE2EDuration="45.422400547s" totalImagesPullingTime="8.430756729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 21:51:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:21.422197525 +0000 UTC m=+82.110173619" watchObservedRunningTime="2026-07-14 21:52:21.422400547 +0000 UTC m=+82.110376640"
Jul 14 21:52:22.344967 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:22.344935    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qgrtx"
Jul 14 21:52:22.866496 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:22.866454    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:22.924884 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:22.924846    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:23.509179 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:23.509139    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:23.608997 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:23.608960    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:23.675278 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:23.675244    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:23.884743 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:23.884651    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:23.972247 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:23.972197    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:24.142431 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:24.142326    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:24.209944 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:24.209908    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:24.319625 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:24.319591    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:24.420911 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:24.420790    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:31.942963 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:31.942927    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:32.004940 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:32.004899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:32.914792 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:32.914753    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:33.015179 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:33.015143    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:33.325241 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:33.325205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:33.427735 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:33.427698    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:33.724491 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:33.724456    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:33.825617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:33.825583    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:34.129015 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:34.128924    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:34.230617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:34.230584    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:35.588179 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:35.588140    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:35.693778 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:35.693738    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:35.762202 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:35.762170    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:35.983575 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:35.983539    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:36.087549 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:36.087514    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:36.287019 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:36.286902    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:37.374473 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:37.374443    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75f9c687fc-2cf7f"
Jul 14 21:52:37.540243 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:37.540205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:37.642752 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:37.642660    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.297744 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.297702    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.506857 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.506797    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.627510 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.627408    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.667296 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.667254    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d6j6p"]
Jul 14 21:52:38.673210 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.673181    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.678143 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.678114    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 21:52:38.678329 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.678156    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 21:52:38.678329 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.678192    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 21:52:38.685724 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.685693    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 21:52:38.685930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.685906    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 21:52:38.686082 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.686067    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 21:52:38.688922 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.688895    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vxhfs\""
Jul 14 21:52:38.758744 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.758703    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.830133 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830093    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-sys\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-metrics-client-ca\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830195    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhpj8\" (UniqueName: \"kubernetes.io/projected/b4bcb077-6179-4b3a-b247-5faf6ef426a4-kube-api-access-bhpj8\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830225    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830255    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830304 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830284    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-root\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830469 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830327    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-textfile\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830469 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830350    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-wtmp\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.830469 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.830384    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.854732 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.854700    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:38.931076 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.930983    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-metrics-client-ca\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931076 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931026    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhpj8\" (UniqueName: \"kubernetes.io/projected/b4bcb077-6179-4b3a-b247-5faf6ef426a4-kube-api-access-bhpj8\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931076 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931054    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931076 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931074    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931105    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-root\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-textfile\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-wtmp\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:52:38.931183    2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931211    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931247    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-sys\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:52:38.931305    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls podName:b4bcb077-6179-4b3a-b247-5faf6ef426a4 nodeName:}" failed. No retries permitted until 2026-07-14 21:52:39.431283158 +0000 UTC m=+100.119259233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls") pod "node-exporter-d6j6p" (UID: "b4bcb077-6179-4b3a-b247-5faf6ef426a4") : secret "node-exporter-tls" not found
Jul 14 21:52:38.931465 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931354    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-root\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931474    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-sys\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-wtmp\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931629    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-textfile\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931656    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-metrics-client-ca\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.931874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.931689    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-accelerators-collector-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.933874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.933849    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:38.941067 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:38.941036    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhpj8\" (UniqueName: \"kubernetes.io/projected/b4bcb077-6179-4b3a-b247-5faf6ef426a4-kube-api-access-bhpj8\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:39.013303 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.013270    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.114051 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.114014    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.188303 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.188272    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.256359 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.256319    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.365607 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.365564    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.435475 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.435420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:39.437916 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.437887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b4bcb077-6179-4b3a-b247-5faf6ef426a4-node-exporter-tls\") pod \"node-exporter-d6j6p\" (UID: \"b4bcb077-6179-4b3a-b247-5faf6ef426a4\") " pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:39.543390 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.543287    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.586351 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.586314    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d6j6p"
Jul 14 21:52:39.599407 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:52:39.599377    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4bcb077_6179_4b3a_b247_5faf6ef426a4.slice/crio-b5c930a182d5f6121c02fc52fed0031ba631b0467841d7e31a307b18cb82406b WatchSource:0}: Error finding container b5c930a182d5f6121c02fc52fed0031ba631b0467841d7e31a307b18cb82406b: Status 404 returned error can't find the container with id b5c930a182d5f6121c02fc52fed0031ba631b0467841d7e31a307b18cb82406b
Jul 14 21:52:39.663131 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.663094    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:39.773848 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:39.773780    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:40.455361 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:40.455323    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6j6p" event={"ID":"b4bcb077-6179-4b3a-b247-5faf6ef426a4","Type":"ContainerStarted","Data":"b5c930a182d5f6121c02fc52fed0031ba631b0467841d7e31a307b18cb82406b"}
Jul 14 21:52:41.462480 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.462440    2570 generic.go:356] "Generic (PLEG): container finished" podID="b4bcb077-6179-4b3a-b247-5faf6ef426a4" containerID="e5af8d7c2bb1b6bd6ae96f3a7b1876fad66a91687b97211afe2a4b7e1363b720" exitCode=0
Jul 14 21:52:41.462944 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.462490    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6j6p" event={"ID":"b4bcb077-6179-4b3a-b247-5faf6ef426a4","Type":"ContainerDied","Data":"e5af8d7c2bb1b6bd6ae96f3a7b1876fad66a91687b97211afe2a4b7e1363b720"}
Jul 14 21:52:41.491632 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.491597    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:41.591901 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.591860    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:41.691551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.691519    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:41.792898 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:41.792855    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:42.266404 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:42.266365    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:42.469150 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:42.469112    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6j6p" event={"ID":"b4bcb077-6179-4b3a-b247-5faf6ef426a4","Type":"ContainerStarted","Data":"9ae11619f108ab70d754475be8120caa834b4ba11b75203b70d9f4a8d9e0b647"}
Jul 14 21:52:42.469537 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:42.469176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d6j6p" event={"ID":"b4bcb077-6179-4b3a-b247-5faf6ef426a4","Type":"ContainerStarted","Data":"143ad803e443b11886665e14742fc5719c425babaeeb81b93699ae2a50777a23"}
Jul 14 21:52:42.492751 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:42.492694    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d6j6p" podStartSLOduration=3.44303831 podStartE2EDuration="4.492674108s" totalImagesPullingTime="937.254768ms" totalInitContainerRuntime="112.38103ms" isStatefulPod=true podCreationTimestamp="2026-07-14 21:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:52:42.491251019 +0000 UTC m=+103.179227156" watchObservedRunningTime="2026-07-14 21:52:42.492674108 +0000 UTC m=+103.180650202"
Jul 14 21:52:43.345077 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:43.345049    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mk8w6"
Jul 14 21:52:43.355372 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:43.355339    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:43.497237 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:43.497200    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:43.636009 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:43.635912    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:43.811117 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:43.811081    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:44.837904 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:44.837867    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:48.664773 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:48.664725    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:48.724075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:48.724040    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:54.296771 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:54.296728    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:54.376189 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:54.376159    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:59.743854 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:59.743798    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:59.836251 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:59.836214    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:52:59.896376 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:52:59.896338    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:53:02.542063 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:02.542025    2570 generic.go:356] "Generic (PLEG): container finished" podID="138a5340-83c5-4570-aeff-d12e703b5342" containerID="cfde6948fd12e323c782e699622dd747a9f23f41ddf56b0749c74f4b125b465d" exitCode=0
Jul 14 21:53:02.542456 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:02.542097    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j" event={"ID":"138a5340-83c5-4570-aeff-d12e703b5342","Type":"ContainerDied","Data":"cfde6948fd12e323c782e699622dd747a9f23f41ddf56b0749c74f4b125b465d"}
Jul 14 21:53:02.542456 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:02.542420    2570 scope.go:117] "RemoveContainer" containerID="cfde6948fd12e323c782e699622dd747a9f23f41ddf56b0749c74f4b125b465d"
Jul 14 21:53:03.548120 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:03.548078    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8h44j" event={"ID":"138a5340-83c5-4570-aeff-d12e703b5342","Type":"ContainerStarted","Data":"571e00fc3c9721a8529bef611a8863b46a5e11152a3379d4e1ac186da2221ea4"}
Jul 14 21:53:09.281243 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:09.281215    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:53:09.486587 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:09.486556    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:53:11.577725 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:11.577691    2570 generic.go:356] "Generic (PLEG): container finished" podID="9f9f61e4-41b3-4e1a-8542-39b4e269327a" containerID="d0350812a5e03ecafdb918fa835dbb418820936acf9f7035c910eb59d025246a" exitCode=0
Jul 14 21:53:11.578110 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:11.577747    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd" event={"ID":"9f9f61e4-41b3-4e1a-8542-39b4e269327a","Type":"ContainerDied","Data":"d0350812a5e03ecafdb918fa835dbb418820936acf9f7035c910eb59d025246a"}
Jul 14 21:53:11.578110 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:11.578059    2570 scope.go:117] "RemoveContainer" containerID="d0350812a5e03ecafdb918fa835dbb418820936acf9f7035c910eb59d025246a"
Jul 14 21:53:12.583490 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:12.583455    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-vsttd" event={"ID":"9f9f61e4-41b3-4e1a-8542-39b4e269327a","Type":"ContainerStarted","Data":"d6feb9b93677ab984d3e10fda8c3b53cae6975f46ac97c595d21ff174403ea02"}
Jul 14 21:53:54.663556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:54.663471    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:53:54.750346 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:53:54.750317    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:00.468176 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:00.468137    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:00.571943 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:00.571911    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:01.138508 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:01.138456    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:01.215978 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:01.215919    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:06.595567 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:06.595527    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:06.696160 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:06.696124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:09.286546 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:09.286503    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:09.390131 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:09.390099    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:09.466440 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:09.466404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:11.064336 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:11.064292    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:11.184466 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:11.184424    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:11.290245 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:11.290210    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:22.938877 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:22.938838    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:23.038738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:23.038695    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:28.549317 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:28.549276    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:29.902091 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:29.902044    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:45.579299 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:45.579253    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:49.635161 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:49.635123    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:54:49.733232 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:54:49.733197    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:11.500263 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:11.500230    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:12.438399 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:12.438362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:15.075356 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:15.075319    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:15.175315 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:15.175279    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:17.118777 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:17.118744    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:17.221412 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:17.221379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:17.540691 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:17.540656    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:17.665045 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:17.664998    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:19.501128 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:19.501089    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:19.917090 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:19.916999    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:22.117612 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:22.117527    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:22.181759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:22.181723    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:26.259584 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:26.259549    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:26.330914 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:26.330879    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:26.727237 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:26.727205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:29.084858 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:29.084823    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:29.180206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:29.180169    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:31.298869 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.298831    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"]
Jul 14 21:55:31.301965 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.301947    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.304529 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.304504    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 21:55:31.304648 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.304541    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 21:55:31.305281 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.305257    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kxml4\""
Jul 14 21:55:31.310605 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.310575    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"]
Jul 14 21:55:31.443462 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.443428    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96kg4\" (UniqueName: \"kubernetes.io/projected/1e8ed256-e76a-4569-8d38-c3f21ba11108-kube-api-access-96kg4\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.443653 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.443478    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e8ed256-e76a-4569-8d38-c3f21ba11108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.544430 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.544397    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96kg4\" (UniqueName: \"kubernetes.io/projected/1e8ed256-e76a-4569-8d38-c3f21ba11108-kube-api-access-96kg4\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.544631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.544447    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e8ed256-e76a-4569-8d38-c3f21ba11108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.544771 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.544749    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e8ed256-e76a-4569-8d38-c3f21ba11108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.553505 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.553447    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96kg4\" (UniqueName: \"kubernetes.io/projected/1e8ed256-e76a-4569-8d38-c3f21ba11108-kube-api-access-96kg4\") pod \"openshift-lws-operator-fd8ccff4c-h7kdq\" (UID: \"1e8ed256-e76a-4569-8d38-c3f21ba11108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.611435 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.611395    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"
Jul 14 21:55:31.738698 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:31.738665    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq"]
Jul 14 21:55:31.741721 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:55:31.741690    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e8ed256_e76a_4569_8d38_c3f21ba11108.slice/crio-33745d3eb60a5d0ba39bd2c034824a6d3b158174d1118ed311e1289849eb668a WatchSource:0}: Error finding container 33745d3eb60a5d0ba39bd2c034824a6d3b158174d1118ed311e1289849eb668a: Status 404 returned error can't find the container with id 33745d3eb60a5d0ba39bd2c034824a6d3b158174d1118ed311e1289849eb668a
Jul 14 21:55:32.073513 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:32.073481    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq" event={"ID":"1e8ed256-e76a-4569-8d38-c3f21ba11108","Type":"ContainerStarted","Data":"33745d3eb60a5d0ba39bd2c034824a6d3b158174d1118ed311e1289849eb668a"}
Jul 14 21:55:35.087948 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:35.087910    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq" event={"ID":"1e8ed256-e76a-4569-8d38-c3f21ba11108","Type":"ContainerStarted","Data":"2847c654194ad24154c68ea63c3ff3471c39b1fe2390cb8e4393ad7fbee1c5d6"}
Jul 14 21:55:35.108384 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:35.108329    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-h7kdq" podStartSLOduration=1.400346722 podStartE2EDuration="4.10831055s" totalImagesPullingTime="2.707963828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:55:35.107384026 +0000 UTC m=+275.795360119" watchObservedRunningTime="2026-07-14 21:55:35.10831055 +0000 UTC m=+275.796286642"
Jul 14 21:55:35.828257 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:35.828219    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:35.929627 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:35.929587    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:37.629529 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:37.629492    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:54.888180 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:54.888141    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:54.995898 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:54.995864    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:55.501589 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:55.501557    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:55.901295 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:55.901190    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.004951 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.004920    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.064778 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.064742    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"]
Jul 14 21:55:56.069017 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.068987    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.071504 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.071478    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8svd6\""
Jul 14 21:55:56.071625 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.071557    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 21:55:56.071625 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.071567    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 21:55:56.071889 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.071875    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 21:55:56.072009 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.071995    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 21:55:56.100002 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.099969    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"]
Jul 14 21:55:56.116418 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.116374    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-tmp\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.116418 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.116420    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.116638 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.116444    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.116638 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.116473    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmq8\" (UniqueName: \"kubernetes.io/projected/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-kube-api-access-7hmq8\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.116638 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.116532    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-manifests\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.216878 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.216840    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-manifests\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217111 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.216891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-tmp\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217111 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.216915    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217111 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.216947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217111 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.216980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmq8\" (UniqueName: \"kubernetes.io/projected/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-kube-api-access-7hmq8\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217367 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.217340    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-manifests\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.217874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.217842    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.219453 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.219423    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-tmp\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.219575 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.219551    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.219632 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.219555    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.227020 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.226994    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmq8\" (UniqueName: \"kubernetes.io/projected/fea1c75a-2dbd-4fe5-8520-2c4de6fa004a-kube-api-access-7hmq8\") pod \"opendatahub-operator-controller-manager-5c99649887-j8pwd\" (UID: \"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a\") " pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.313061 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.313002    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.379264 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.379221    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:55:56.414872 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.414836    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.491361 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.491276    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:56.541967 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.540621    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"]
Jul 14 21:55:56.550132 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:55:56.550097    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea1c75a_2dbd_4fe5_8520_2c4de6fa004a.slice/crio-0df52dce7a0b778323c926e3d156aa074bd757ce8ea7d80b1c16120b23f68d98 WatchSource:0}: Error finding container 0df52dce7a0b778323c926e3d156aa074bd757ce8ea7d80b1c16120b23f68d98: Status 404 returned error can't find the container with id 0df52dce7a0b778323c926e3d156aa074bd757ce8ea7d80b1c16120b23f68d98
Jul 14 21:55:56.651049 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:56.651009    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:57.166401 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:57.166366    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd" event={"ID":"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a","Type":"ContainerStarted","Data":"0df52dce7a0b778323c926e3d156aa074bd757ce8ea7d80b1c16120b23f68d98"}
Jul 14 21:55:57.934132 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:57.934089    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:58.064412 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:58.064341    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:55:59.780767 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:59.780741    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 21:55:59.781173 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:59.780782    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 21:55:59.795683 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:59.795652    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:55:59.795982 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:59.795958    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 21:55:59.799434 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:55:59.799412    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 21:56:00.181316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:00.181216    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd" event={"ID":"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a","Type":"ContainerStarted","Data":"14575f8ae53487d2a4d327402441e186aab66d7b6c6681f0a762c4d420440e3a"}
Jul 14 21:56:01.186164 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:01.186130    2570 generic.go:356] "Generic (PLEG): container finished" podID="fea1c75a-2dbd-4fe5-8520-2c4de6fa004a" containerID="14575f8ae53487d2a4d327402441e186aab66d7b6c6681f0a762c4d420440e3a" exitCode=0
Jul 14 21:56:01.186643 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:01.186172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd" event={"ID":"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a","Type":"ContainerDied","Data":"14575f8ae53487d2a4d327402441e186aab66d7b6c6681f0a762c4d420440e3a"}
Jul 14 21:56:01.186787 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:01.186768    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 21:56:03.195200 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:03.195166    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd" event={"ID":"fea1c75a-2dbd-4fe5-8520-2c4de6fa004a","Type":"ContainerStarted","Data":"1c58f6b2c8cd00fe623cc568cbcae12f008b4eeef570bbbfee2fc144bea05571"}
Jul 14 21:56:03.195606 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:03.195271    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:56:03.227758 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:03.227697    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd" podStartSLOduration=1.47541814 podStartE2EDuration="7.22767693s" totalImagesPullingTime="4.599843002s" totalInitContainerRuntime="1.152415788s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:55:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:56:03.225572581 +0000 UTC m=+303.913548693" watchObservedRunningTime="2026-07-14 21:56:03.22767693 +0000 UTC m=+303.915653022"
Jul 14 21:56:08.401262 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:08.401212    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:10.222272 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:10.222237    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:14.201575 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:14.201546    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c99649887-j8pwd"
Jul 14 21:56:15.779228 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:15.779187    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:15.903219 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:15.903182    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:16.421419 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.421377    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"]
Jul 14 21:56:16.429331 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.429307    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.435233 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.435208    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 21:56:16.436139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.436112    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 21:56:16.436320 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.436304    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 21:56:16.438568 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.438546    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 21:56:16.454611 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.454578    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"]
Jul 14 21:56:16.483786 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.483751    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tmp\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.483998 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.483846    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tls-certs\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.483998 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.483881    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw4f7\" (UniqueName: \"kubernetes.io/projected/e4db6d7b-83b1-4af9-9684-a5d87c74d924-kube-api-access-jw4f7\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.584851 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.584789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tls-certs\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.585059 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.584862    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw4f7\" (UniqueName: \"kubernetes.io/projected/e4db6d7b-83b1-4af9-9684-a5d87c74d924-kube-api-access-jw4f7\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.585059 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.584899    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tmp\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.587228 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.587197    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tmp\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.587439 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.587418    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e4db6d7b-83b1-4af9-9684-a5d87c74d924-tls-certs\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.608316 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.608288    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw4f7\" (UniqueName: \"kubernetes.io/projected/e4db6d7b-83b1-4af9-9684-a5d87c74d924-kube-api-access-jw4f7\") pod \"kube-auth-proxy-9dcf98947-d5nvn\" (UID: \"e4db6d7b-83b1-4af9-9684-a5d87c74d924\") " pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.738832 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.738777    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"
Jul 14 21:56:16.886591 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:16.886552    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn"]
Jul 14 21:56:16.890184 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:56:16.890156    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4db6d7b_83b1_4af9_9684_a5d87c74d924.slice/crio-1c8365241cbcd887dcaa729519c1c62061a7a58d8763f75e932861dd0d04c871 WatchSource:0}: Error finding container 1c8365241cbcd887dcaa729519c1c62061a7a58d8763f75e932861dd0d04c871: Status 404 returned error can't find the container with id 1c8365241cbcd887dcaa729519c1c62061a7a58d8763f75e932861dd0d04c871
Jul 14 21:56:17.141689 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:17.141556    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:17.248321 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:17.248280    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn" event={"ID":"e4db6d7b-83b1-4af9-9684-a5d87c74d924","Type":"ContainerStarted","Data":"1c8365241cbcd887dcaa729519c1c62061a7a58d8763f75e932861dd0d04c871"}
Jul 14 21:56:18.217696 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.217656    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:18.333832 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.333763    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:18.541653 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.541550    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:18.541884 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.541865    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:18.545144 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.545123    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.548480 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.548457    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 14 21:56:18.548480 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.548464    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 14 21:56:18.548776 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.548760    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f7fdj\""
Jul 14 21:56:18.548886 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.548822    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 14 21:56:18.549044 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.549029    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 14 21:56:18.560143 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.560109    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:18.601784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.601742    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.601995 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.601794    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.601995 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.601883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzsgk\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.601995 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.601935    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.601995 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.601977    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.602155 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.602088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.602155 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.602125    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.634491 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.634456    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:18.703126 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703094    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703260 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703139    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703260 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703260 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703260 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703209    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703260 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703253    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzsgk\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.703497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.703291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.704094 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.704039    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.705761 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.705733    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.705761 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.705754    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.705982 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.705736    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.705982 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.705843    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.711520 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.711481    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.711765 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.711747    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzsgk\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk\") pod \"istiod-openshift-gateway-7574459656-9t6mb\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:18.855140 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:18.855050    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:19.541491 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:19.541461    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:19.542780 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:56:19.542753    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd430bc3_b04c_409b_be00_48b7f25fde62.slice/crio-02e851af651d6ca17db1011092e0053633aa4ac4b2991f4db16d6858ad7a0714 WatchSource:0}: Error finding container 02e851af651d6ca17db1011092e0053633aa4ac4b2991f4db16d6858ad7a0714: Status 404 returned error can't find the container with id 02e851af651d6ca17db1011092e0053633aa4ac4b2991f4db16d6858ad7a0714
Jul 14 21:56:19.720697 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:19.720659    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:20.263566 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:20.263531    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn" event={"ID":"e4db6d7b-83b1-4af9-9684-a5d87c74d924","Type":"ContainerStarted","Data":"203f2a2f484a81df74d6ab991c37215d46ed12982746ee37eab94a9981f216ce"}
Jul 14 21:56:20.264764 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:20.264738    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" event={"ID":"bd430bc3-b04c-409b-be00-48b7f25fde62","Type":"ContainerStarted","Data":"02e851af651d6ca17db1011092e0053633aa4ac4b2991f4db16d6858ad7a0714"}
Jul 14 21:56:20.281270 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:20.281200    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-9dcf98947-d5nvn" podStartSLOduration=1.73107649 podStartE2EDuration="4.281182225s" totalImagesPullingTime="2.550105735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:56:20.280424855 +0000 UTC m=+320.968400947" watchObservedRunningTime="2026-07-14 21:56:20.281182225 +0000 UTC m=+320.969158389"
Jul 14 21:56:23.053257 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.053219    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 14 21:56:23.053595 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.053285    2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 14 21:56:23.278848 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.278785    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" event={"ID":"bd430bc3-b04c-409b-be00-48b7f25fde62","Type":"ContainerStarted","Data":"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"}
Jul 14 21:56:23.279045 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.278872    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:23.307080 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.306970    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" podStartSLOduration=1.798813316 podStartE2EDuration="5.306952202s" totalImagesPullingTime="3.508138886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:56:23.304594707 +0000 UTC m=+323.992570872" watchObservedRunningTime="2026-07-14 21:56:23.306952202 +0000 UTC m=+323.994928295"
Jul 14 21:56:23.793397 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.793333    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:23.935164 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:23.935118    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:24.170648 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:24.170551    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:24.285121 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:24.284910    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:24.350042 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:24.350009    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:24.621225 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:24.621184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:26.905098 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.905065    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:26.982634 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.982595    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-269c5"]
Jul 14 21:56:26.986456 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.986426    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:26.989258 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.989228    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 14 21:56:26.989560 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.989298    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wf7lc\""
Jul 14 21:56:26.989560 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.989337    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"]
Jul 14 21:56:26.992743 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.992720    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:26.995631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.995590    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 14 21:56:26.995770 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:26.995731    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9srf4\""
Jul 14 21:56:27.003291 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.003260    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-269c5"]
Jul 14 21:56:27.016941 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.016899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:27.018733 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.018702    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"]
Jul 14 21:56:27.079751 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.079713    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h24xt\" (UniqueName: \"kubernetes.io/projected/4b9faa79-d47a-4a43-97ea-27175eaecfc3-kube-api-access-h24xt\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.079751 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.079753    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9faa79-d47a-4a43-97ea-27175eaecfc3-cert\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.080078 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.079785    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62f669f2-7166-41c7-b4f7-fef4275cbfdd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.080078 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.079897    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddmt2\" (UniqueName: \"kubernetes.io/projected/62f669f2-7166-41c7-b4f7-fef4275cbfdd-kube-api-access-ddmt2\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.180561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.180469    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h24xt\" (UniqueName: \"kubernetes.io/projected/4b9faa79-d47a-4a43-97ea-27175eaecfc3-kube-api-access-h24xt\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.180561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.180511    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9faa79-d47a-4a43-97ea-27175eaecfc3-cert\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.180561 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.180535    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62f669f2-7166-41c7-b4f7-fef4275cbfdd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.180781 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.180576    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddmt2\" (UniqueName: \"kubernetes.io/projected/62f669f2-7166-41c7-b4f7-fef4275cbfdd-kube-api-access-ddmt2\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.183100 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.183073    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62f669f2-7166-41c7-b4f7-fef4275cbfdd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.183224 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.183081    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b9faa79-d47a-4a43-97ea-27175eaecfc3-cert\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.202935 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.202898    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h24xt\" (UniqueName: \"kubernetes.io/projected/4b9faa79-d47a-4a43-97ea-27175eaecfc3-kube-api-access-h24xt\") pod \"kserve-controller-manager-d6f4b65b8-269c5\" (UID: \"4b9faa79-d47a-4a43-97ea-27175eaecfc3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.208518 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.208488    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddmt2\" (UniqueName: \"kubernetes.io/projected/62f669f2-7166-41c7-b4f7-fef4275cbfdd-kube-api-access-ddmt2\") pod \"llmisvc-controller-manager-777c6bd7d5-zqm5f\" (UID: \"62f669f2-7166-41c7-b4f7-fef4275cbfdd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.217354 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.217310    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:27.303986 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.303945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:27.322534 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.322496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:27.423735 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.423704    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:27.466637 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.466614    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-269c5"]
Jul 14 21:56:27.469043 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:56:27.469014    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9faa79_d47a_4a43_97ea_27175eaecfc3.slice/crio-c59387ece3887f316b15b90d95b23a7ec60560beb74b13c77bdb6713593f2267 WatchSource:0}: Error finding container c59387ece3887f316b15b90d95b23a7ec60560beb74b13c77bdb6713593f2267: Status 404 returned error can't find the container with id c59387ece3887f316b15b90d95b23a7ec60560beb74b13c77bdb6713593f2267
Jul 14 21:56:27.490016 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:56:27.489986    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod62f669f2_7166_41c7_b4f7_fef4275cbfdd.slice/crio-747f92994a1ef4afc049be25a1010d7db0ed47903971e3634d8e116bb5386f39 WatchSource:0}: Error finding container 747f92994a1ef4afc049be25a1010d7db0ed47903971e3634d8e116bb5386f39: Status 404 returned error can't find the container with id 747f92994a1ef4afc049be25a1010d7db0ed47903971e3634d8e116bb5386f39
Jul 14 21:56:27.496105 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.496065    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"]
Jul 14 21:56:27.542738 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:27.542701    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:56:28.301888 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:28.301854    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f" event={"ID":"62f669f2-7166-41c7-b4f7-fef4275cbfdd","Type":"ContainerStarted","Data":"747f92994a1ef4afc049be25a1010d7db0ed47903971e3634d8e116bb5386f39"}
Jul 14 21:56:28.303287 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:28.303261    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5" event={"ID":"4b9faa79-d47a-4a43-97ea-27175eaecfc3","Type":"ContainerStarted","Data":"c59387ece3887f316b15b90d95b23a7ec60560beb74b13c77bdb6713593f2267"}
Jul 14 21:56:30.591188 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:30.591147    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:30.591749 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:30.591383    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" podUID="bd430bc3-b04c-409b-be00-48b7f25fde62" containerName="discovery" containerID="cri-o://2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003" gracePeriod=30
Jul 14 21:56:30.883460 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:30.883432    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:31.014565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014475    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzsgk\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014565 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014542    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014791 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014573    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014791 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014622    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014791 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014648    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014979 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014884    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.014979 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.014933    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig\") pod \"bd430bc3-b04c-409b-be00-48b7f25fde62\" (UID: \"bd430bc3-b04c-409b-be00-48b7f25fde62\") "
Jul 14 21:56:31.015455 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.015338    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 21:56:31.017876 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.017753    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 21:56:31.017876 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.017678    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs" (OuterVolumeSpecName: "local-certs") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 21:56:31.018994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.018066    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk" (OuterVolumeSpecName: "kube-api-access-hzsgk") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "kube-api-access-hzsgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:56:31.018994 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.018256    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:56:31.019784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.019743    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 21:56:31.020484 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.020457    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts" (OuterVolumeSpecName: "cacerts") pod "bd430bc3-b04c-409b-be00-48b7f25fde62" (UID: "bd430bc3-b04c-409b-be00-48b7f25fde62"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 21:56:31.116507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116465    2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-token\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116507 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116503    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-dns-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116518    2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-cacerts\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116531    2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-csr-ca-configmap\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116542    2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bd430bc3-b04c-409b-be00-48b7f25fde62-istio-kubeconfig\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116552    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzsgk\" (UniqueName: \"kubernetes.io/projected/bd430bc3-b04c-409b-be00-48b7f25fde62-kube-api-access-hzsgk\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.116712 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.116563    2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd430bc3-b04c-409b-be00-48b7f25fde62-local-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:56:31.319857 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.319746    2570 generic.go:356] "Generic (PLEG): container finished" podID="bd430bc3-b04c-409b-be00-48b7f25fde62" containerID="2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003" exitCode=0
Jul 14 21:56:31.319857 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.319795    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"
Jul 14 21:56:31.320128 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.319842    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" event={"ID":"bd430bc3-b04c-409b-be00-48b7f25fde62","Type":"ContainerDied","Data":"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"}
Jul 14 21:56:31.320128 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.319912    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb" event={"ID":"bd430bc3-b04c-409b-be00-48b7f25fde62","Type":"ContainerDied","Data":"02e851af651d6ca17db1011092e0053633aa4ac4b2991f4db16d6858ad7a0714"}
Jul 14 21:56:31.320128 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.319941    2570 scope.go:117] "RemoveContainer" containerID="2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"
Jul 14 21:56:31.322078 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.322046    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5" event={"ID":"4b9faa79-d47a-4a43-97ea-27175eaecfc3","Type":"ContainerStarted","Data":"b62b6b2c6a5dfa94330825defe1cc55f19ec1fbf6d201222eecdde92a0158e4c"}
Jul 14 21:56:31.322212 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.322200    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:56:31.343340 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.343278    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5" podStartSLOduration=2.304274573 podStartE2EDuration="5.343253803s" totalImagesPullingTime="3.03897923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:56:31.341626057 +0000 UTC m=+332.029602169" watchObservedRunningTime="2026-07-14 21:56:31.343253803 +0000 UTC m=+332.031229899"
Jul 14 21:56:31.361046 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.361008    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:31.367762 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.367714    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9t6mb"]
Jul 14 21:56:31.488946 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.488780    2570 scope.go:117] "RemoveContainer" containerID="2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"
Jul 14 21:56:31.489160 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:56:31.489138    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003\": container with ID starting with 2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003 not found: ID does not exist" containerID="2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"
Jul 14 21:56:31.489198 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.489169    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003"} err="failed to get container status \"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003\": rpc error: code = NotFound desc = could not find container \"2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003\": container with ID starting with 2a4833da175a2ee88e0c3bdce6441e373178ad6bd5688ad48578ffafd39f8003 not found: ID does not exist"
Jul 14 21:56:31.874788 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:31.874704    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd430bc3-b04c-409b-be00-48b7f25fde62" path="/var/lib/kubelet/pods/bd430bc3-b04c-409b-be00-48b7f25fde62/volumes"
Jul 14 21:56:32.328345 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:32.328308    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f" event={"ID":"62f669f2-7166-41c7-b4f7-fef4275cbfdd","Type":"ContainerStarted","Data":"25dd226be5db6611a19bd523de14cbd0d9fcf3e96371217e8e5288b8eff6c30f"}
Jul 14 21:56:32.328544 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:32.328473    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:56:32.348063 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:56:32.347984    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f" podStartSLOduration=2.301559493 podStartE2EDuration="6.347965051s" totalImagesPullingTime="4.046405558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:56:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:56:32.346677091 +0000 UTC m=+333.034653195" watchObservedRunningTime="2026-07-14 21:56:32.347965051 +0000 UTC m=+333.035941143"
Jul 14 21:57:02.334798 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:02.334760    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-269c5"
Jul 14 21:57:03.336205 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:03.336173    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zqm5f"
Jul 14 21:57:17.344088 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.344051    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:17.488290 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.488256    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:17.507943 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.507909    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:17.508206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.508194    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd430bc3-b04c-409b-be00-48b7f25fde62" containerName="discovery"
Jul 14 21:57:17.508206 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.508208    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd430bc3-b04c-409b-be00-48b7f25fde62" containerName="discovery"
Jul 14 21:57:17.508287 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.508277    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd430bc3-b04c-409b-be00-48b7f25fde62" containerName="discovery"
Jul 14 21:57:17.520303 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.520274    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:17.524146 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.524114    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-w9b6n\""
Jul 14 21:57:17.524311 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.524122    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 21:57:17.524311 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.524121    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 21:57:17.560648 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.560617    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:17.583642 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.583605    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrvlt\" (UniqueName: \"kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt\") pod \"kuadrant-operator-catalog-hcv76\" (UID: \"2feb8d5a-6843-49a3-8581-15f406a9473c\") " pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:17.655688 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.655589    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:17.684543 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.684508    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrvlt\" (UniqueName: \"kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt\") pod \"kuadrant-operator-catalog-hcv76\" (UID: \"2feb8d5a-6843-49a3-8581-15f406a9473c\") " pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:17.708678 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.708648    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrvlt\" (UniqueName: \"kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt\") pod \"kuadrant-operator-catalog-hcv76\" (UID: \"2feb8d5a-6843-49a3-8581-15f406a9473c\") " pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:17.830902 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.830850    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:17.841126 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.841093    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:17.984325 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:17.984292    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:18.050682 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.050644    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-brfqp"]
Jul 14 21:57:18.055366 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.055330    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:18.069861 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.069824    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-brfqp"]
Jul 14 21:57:18.088722 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.088676    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnqt\" (UniqueName: \"kubernetes.io/projected/12312a97-d4bf-4755-8fad-6cf665811b32-kube-api-access-msnqt\") pod \"kuadrant-operator-catalog-brfqp\" (UID: \"12312a97-d4bf-4755-8fad-6cf665811b32\") " pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:18.189648 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.189616    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnqt\" (UniqueName: \"kubernetes.io/projected/12312a97-d4bf-4755-8fad-6cf665811b32-kube-api-access-msnqt\") pod \"kuadrant-operator-catalog-brfqp\" (UID: \"12312a97-d4bf-4755-8fad-6cf665811b32\") " pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:18.222843 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.222790    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnqt\" (UniqueName: \"kubernetes.io/projected/12312a97-d4bf-4755-8fad-6cf665811b32-kube-api-access-msnqt\") pod \"kuadrant-operator-catalog-brfqp\" (UID: \"12312a97-d4bf-4755-8fad-6cf665811b32\") " pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:18.298634 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.298599    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:18.365351 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.365301    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:18.508341 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.508306    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:18.525014 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.524978    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" event={"ID":"2feb8d5a-6843-49a3-8581-15f406a9473c","Type":"ContainerStarted","Data":"4d9bb4e3690721e338b7d902bf1768272c5d44ee92ef08296ec072516cc09cf4"}
Jul 14 21:57:18.532057 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:18.532023    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-brfqp"]
Jul 14 21:57:18.535730 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:57:18.535703    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12312a97_d4bf_4755_8fad_6cf665811b32.slice/crio-c0a7e87845086f4d5e237270560f3e3d7be3200879d769581c60f0e27afab395 WatchSource:0}: Error finding container c0a7e87845086f4d5e237270560f3e3d7be3200879d769581c60f0e27afab395: Status 404 returned error can't find the container with id c0a7e87845086f4d5e237270560f3e3d7be3200879d769581c60f0e27afab395
Jul 14 21:57:19.531765 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:19.531725    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-brfqp" event={"ID":"12312a97-d4bf-4755-8fad-6cf665811b32","Type":"ContainerStarted","Data":"c0a7e87845086f4d5e237270560f3e3d7be3200879d769581c60f0e27afab395"}
Jul 14 21:57:20.538361 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.538257    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" event={"ID":"2feb8d5a-6843-49a3-8581-15f406a9473c","Type":"ContainerStarted","Data":"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"}
Jul 14 21:57:20.538361 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.538307    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" podUID="2feb8d5a-6843-49a3-8581-15f406a9473c" containerName="registry-server" containerID="cri-o://f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777" gracePeriod=2
Jul 14 21:57:20.539865 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.539837    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-brfqp" event={"ID":"12312a97-d4bf-4755-8fad-6cf665811b32","Type":"ContainerStarted","Data":"3fb35e41c146b9bd9f4f5b3580c96da7a04a33de8740ea75f0ec4bb46eb78dbc"}
Jul 14 21:57:20.572031 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.571970    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" podStartSLOduration=1.343546027 podStartE2EDuration="3.57194909s" totalImagesPullingTime="2.228403063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:57:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:57:20.570884029 +0000 UTC m=+381.258860123" watchObservedRunningTime="2026-07-14 21:57:20.57194909 +0000 UTC m=+381.259925201"
Jul 14 21:57:20.599245 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.599183    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-brfqp" podStartSLOduration=0.916883747 podStartE2EDuration="2.599166929s" totalImagesPullingTime="1.682283182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:57:20.597113852 +0000 UTC m=+381.285089966" watchObservedRunningTime="2026-07-14 21:57:20.599166929 +0000 UTC m=+381.287143056"
Jul 14 21:57:20.794379 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.794297    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:20.914072 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.914032    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrvlt\" (UniqueName: \"kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt\") pod \"2feb8d5a-6843-49a3-8581-15f406a9473c\" (UID: \"2feb8d5a-6843-49a3-8581-15f406a9473c\") "
Jul 14 21:57:20.916291 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:20.916261    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt" (OuterVolumeSpecName: "kube-api-access-xrvlt") pod "2feb8d5a-6843-49a3-8581-15f406a9473c" (UID: "2feb8d5a-6843-49a3-8581-15f406a9473c"). InnerVolumeSpecName "kube-api-access-xrvlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:57:21.015098 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.015058    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrvlt\" (UniqueName: \"kubernetes.io/projected/2feb8d5a-6843-49a3-8581-15f406a9473c-kube-api-access-xrvlt\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:57:21.546851 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.546744    2570 generic.go:356] "Generic (PLEG): container finished" podID="2feb8d5a-6843-49a3-8581-15f406a9473c" containerID="f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777" exitCode=0
Jul 14 21:57:21.546851 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.546797    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hcv76"
Jul 14 21:57:21.546851 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.546827    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" event={"ID":"2feb8d5a-6843-49a3-8581-15f406a9473c","Type":"ContainerDied","Data":"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"}
Jul 14 21:57:21.547432 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.546878    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hcv76" event={"ID":"2feb8d5a-6843-49a3-8581-15f406a9473c","Type":"ContainerDied","Data":"4d9bb4e3690721e338b7d902bf1768272c5d44ee92ef08296ec072516cc09cf4"}
Jul 14 21:57:21.547432 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.546898    2570 scope.go:117] "RemoveContainer" containerID="f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"
Jul 14 21:57:21.555365 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.555346    2570 scope.go:117] "RemoveContainer" containerID="f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"
Jul 14 21:57:21.555663 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:57:21.555647    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777\": container with ID starting with f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777 not found: ID does not exist" containerID="f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"
Jul 14 21:57:21.555728 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.555678    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777"} err="failed to get container status \"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777\": rpc error: code = NotFound desc = could not find container \"f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777\": container with ID starting with f7eeaa02945fdc7cd0f774bb7dedaadd9939e63df9f40503b1f503bc6cfb2777 not found: ID does not exist"
Jul 14 21:57:21.575502 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.575469    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:21.578668 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.578638    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hcv76"]
Jul 14 21:57:21.875049 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:21.874963    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2feb8d5a-6843-49a3-8581-15f406a9473c" path="/var/lib/kubelet/pods/2feb8d5a-6843-49a3-8581-15f406a9473c/volumes"
Jul 14 21:57:28.365904 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:28.365861    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:28.366322 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:28.365924    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:28.389458 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:28.389427    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:28.599196 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:28.599169    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-brfqp"
Jul 14 21:57:39.190527 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:39.190476    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:51.050937 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:51.050898    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:52.028294 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:52.028260    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:55.748849 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.748796    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"]
Jul 14 21:57:55.749340 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.749249    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2feb8d5a-6843-49a3-8581-15f406a9473c" containerName="registry-server"
Jul 14 21:57:55.749340 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.749268    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2feb8d5a-6843-49a3-8581-15f406a9473c" containerName="registry-server"
Jul 14 21:57:55.749340 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.749336    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2feb8d5a-6843-49a3-8581-15f406a9473c" containerName="registry-server"
Jul 14 21:57:55.756982 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.756945    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:55.760120 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.760088    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-czb6w\""
Jul 14 21:57:55.767655 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.767625    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"]
Jul 14 21:57:55.806049 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.806016    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsmfd\" (UniqueName: \"kubernetes.io/projected/f0f332b6-2b68-4e8f-89eb-92825590ad06-kube-api-access-lsmfd\") pod \"authorino-operator-75cc8d5cb7-4hcsb\" (UID: \"f0f332b6-2b68-4e8f-89eb-92825590ad06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:55.906784 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.906744    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsmfd\" (UniqueName: \"kubernetes.io/projected/f0f332b6-2b68-4e8f-89eb-92825590ad06-kube-api-access-lsmfd\") pod \"authorino-operator-75cc8d5cb7-4hcsb\" (UID: \"f0f332b6-2b68-4e8f-89eb-92825590ad06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:55.921823 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:55.921777    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsmfd\" (UniqueName: \"kubernetes.io/projected/f0f332b6-2b68-4e8f-89eb-92825590ad06-kube-api-access-lsmfd\") pod \"authorino-operator-75cc8d5cb7-4hcsb\" (UID: \"f0f332b6-2b68-4e8f-89eb-92825590ad06\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:56.067706 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:56.067605    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:56.218634 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:56.218600    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"]
Jul 14 21:57:56.222107 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:57:56.222072    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f332b6_2b68_4e8f_89eb_92825590ad06.slice/crio-1f616166e2a567305b755026be54138da874e659e0e66b08eec417b50937adee WatchSource:0}: Error finding container 1f616166e2a567305b755026be54138da874e659e0e66b08eec417b50937adee: Status 404 returned error can't find the container with id 1f616166e2a567305b755026be54138da874e659e0e66b08eec417b50937adee
Jul 14 21:57:56.688250 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:56.688215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb" event={"ID":"f0f332b6-2b68-4e8f-89eb-92825590ad06","Type":"ContainerStarted","Data":"1f616166e2a567305b755026be54138da874e659e0e66b08eec417b50937adee"}
Jul 14 21:57:57.050178 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:57.050086    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:58.700399 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:58.700364    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb" event={"ID":"f0f332b6-2b68-4e8f-89eb-92825590ad06","Type":"ContainerStarted","Data":"573f2c962c8d3207b2e46aed01e5bec52ed8b74bb30fdacdbffdea6910db2e31"}
Jul 14 21:57:58.700907 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:58.700542    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:57:58.747691 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:58.747635    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb" podStartSLOduration=1.452577026 podStartE2EDuration="3.747619192s" totalImagesPullingTime="2.295042166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:57:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:57:58.746906677 +0000 UTC m=+419.434882809" watchObservedRunningTime="2026-07-14 21:57:58.747619192 +0000 UTC m=+419.435595285"
Jul 14 21:57:59.553528 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:59.553488    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:57:59.746645 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:57:59.746608    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:01.443433 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.443394    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:01.593768 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.593730    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:01.635067 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.635028    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"]
Jul 14 21:58:01.639955 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.639928    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.642927 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.642902    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5pgl8\""
Jul 14 21:58:01.668918 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.668884    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"]
Jul 14 21:58:01.761196 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.761099    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.761196 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.761134    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmfh5\" (UniqueName: \"kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.862404 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.862370    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.862404 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.862405    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmfh5\" (UniqueName: \"kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.862869 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.862847    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.881018 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.880985    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmfh5\" (UniqueName: \"kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:01.950955 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:01.950913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:02.133514 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:02.133476    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"]
Jul 14 21:58:02.142602 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:02.142559    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0a3b16_389b_4df3_bad0_2f45a6e4e1ed.slice/crio-a75b80773b0518294ed299a13db25faee4ff46ef554841886d4c284f0ce19040 WatchSource:0}: Error finding container a75b80773b0518294ed299a13db25faee4ff46ef554841886d4c284f0ce19040: Status 404 returned error can't find the container with id a75b80773b0518294ed299a13db25faee4ff46ef554841886d4c284f0ce19040
Jul 14 21:58:02.258912 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:02.258869    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:02.610267 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:02.610233    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:02.718848 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:02.718791    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n" event={"ID":"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed","Type":"ContainerStarted","Data":"a75b80773b0518294ed299a13db25faee4ff46ef554841886d4c284f0ce19040"}
Jul 14 21:58:04.454097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.454060    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"]
Jul 14 21:58:04.457430 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.457406    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:04.465185 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.465161    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-xl6z8\""
Jul 14 21:58:04.465752 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.465729    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 14 21:58:04.501059 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.501024    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"]
Jul 14 21:58:04.590769 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.590732    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htfgx\" (UniqueName: \"kubernetes.io/projected/1f9380f5-87b8-49d7-b119-4d8232929d41-kube-api-access-htfgx\") pod \"dns-operator-controller-manager-864dd78c4c-9xn5m\" (UID: \"1f9380f5-87b8-49d7-b119-4d8232929d41\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:04.692343 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.692288    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htfgx\" (UniqueName: \"kubernetes.io/projected/1f9380f5-87b8-49d7-b119-4d8232929d41-kube-api-access-htfgx\") pod \"dns-operator-controller-manager-864dd78c4c-9xn5m\" (UID: \"1f9380f5-87b8-49d7-b119-4d8232929d41\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:04.709047 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.708927    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htfgx\" (UniqueName: \"kubernetes.io/projected/1f9380f5-87b8-49d7-b119-4d8232929d41-kube-api-access-htfgx\") pod \"dns-operator-controller-manager-864dd78c4c-9xn5m\" (UID: \"1f9380f5-87b8-49d7-b119-4d8232929d41\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:04.768062 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.768018    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:04.896551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.896514    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:04.970429 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:04.970407    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"]
Jul 14 21:58:04.972322 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:04.972293    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f9380f5_87b8_49d7_b119_4d8232929d41.slice/crio-86528004387dfebc1f1491ab07e0a3053ddf4df857e685443bfacb03a1653239 WatchSource:0}: Error finding container 86528004387dfebc1f1491ab07e0a3053ddf4df857e685443bfacb03a1653239: Status 404 returned error can't find the container with id 86528004387dfebc1f1491ab07e0a3053ddf4df857e685443bfacb03a1653239
Jul 14 21:58:05.735441 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:05.735399    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m" event={"ID":"1f9380f5-87b8-49d7-b119-4d8232929d41","Type":"ContainerStarted","Data":"86528004387dfebc1f1491ab07e0a3053ddf4df857e685443bfacb03a1653239"}
Jul 14 21:58:05.791604 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:05.791481    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:06.691166 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.691126    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"]
Jul 14 21:58:06.695152 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.695126    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:06.700480 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.700446    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-qwmw7\""
Jul 14 21:58:06.743142 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.743107    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"]
Jul 14 21:58:06.811248 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.811212    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64h44\" (UniqueName: \"kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44\") pod \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" (UID: \"27d63ac9-f826-4dfe-a5d2-36cb78430406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:06.912760 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.912718    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64h44\" (UniqueName: \"kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44\") pod \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" (UID: \"27d63ac9-f826-4dfe-a5d2-36cb78430406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:06.960969 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:06.960887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64h44\" (UniqueName: \"kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44\") pod \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" (UID: \"27d63ac9-f826-4dfe-a5d2-36cb78430406\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:07.008891 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:07.008846    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:07.991867 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:07.991828    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"]
Jul 14 21:58:07.996136 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:07.996091    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d63ac9_f826_4dfe_a5d2_36cb78430406.slice/crio-ed988f801a9ed0da4af0e223ef3cc14030440438b5054bb920c19dd907a792fb WatchSource:0}: Error finding container ed988f801a9ed0da4af0e223ef3cc14030440438b5054bb920c19dd907a792fb: Status 404 returned error can't find the container with id ed988f801a9ed0da4af0e223ef3cc14030440438b5054bb920c19dd907a792fb
Jul 14 21:58:08.056388 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.056348    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:08.450193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.450160    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:08.701335 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.701250    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:08.751523 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.751449    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n" event={"ID":"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed","Type":"ContainerStarted","Data":"2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b"}
Jul 14 21:58:08.751777 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.751729    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:08.753370 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.753341    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m" event={"ID":"1f9380f5-87b8-49d7-b119-4d8232929d41","Type":"ContainerStarted","Data":"cbdfa9bbbc5feceda2a609a2c9937dfc30abf9272e4d7239835e45adfb979439"}
Jul 14 21:58:08.753497 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.753455    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:08.754930 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.754899    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx" event={"ID":"27d63ac9-f826-4dfe-a5d2-36cb78430406","Type":"ContainerStarted","Data":"ed988f801a9ed0da4af0e223ef3cc14030440438b5054bb920c19dd907a792fb"}
Jul 14 21:58:08.799264 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.799205    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n" podStartSLOduration=2.108463428 podStartE2EDuration="7.799188787s" totalImagesPullingTime="5.690725359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:08.793725861 +0000 UTC m=+429.481701964" watchObservedRunningTime="2026-07-14 21:58:08.799188787 +0000 UTC m=+429.487164879"
Jul 14 21:58:08.826830 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.826683    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:08.827566 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:08.827469    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m" podStartSLOduration=1.90097714 podStartE2EDuration="4.8274502s" totalImagesPullingTime="2.92647306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:08.825234043 +0000 UTC m=+429.513210170" watchObservedRunningTime="2026-07-14 21:58:08.8274502 +0000 UTC m=+429.515426294"
Jul 14 21:58:09.115163 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.115069    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:09.209223 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.209183    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:09.708631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.708601    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4hcsb"
Jul 14 21:58:09.760590 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.760555    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx" event={"ID":"27d63ac9-f826-4dfe-a5d2-36cb78430406","Type":"ContainerStarted","Data":"db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734"}
Jul 14 21:58:09.760788 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.760757    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:09.827734 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:09.827544    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx" podStartSLOduration=2.194776943 podStartE2EDuration="3.827524465s" totalImagesPullingTime="1.632747522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:09.826082462 +0000 UTC m=+430.514058554" watchObservedRunningTime="2026-07-14 21:58:09.827524465 +0000 UTC m=+430.515500560"
Jul 14 21:58:11.579899 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.579866    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"]
Jul 14 21:58:11.583263 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.583236    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.591817 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.591780    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 14 21:58:11.591961 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.591780    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 14 21:58:11.601718 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.601691    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k4kj9\""
Jul 14 21:58:11.613236 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.613199    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:11.624765 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.624734    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"]
Jul 14 21:58:11.656934 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.656899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwzw8\" (UniqueName: \"kubernetes.io/projected/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-kube-api-access-vwzw8\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.657139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.657008    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.657139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.657065    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.757924 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.757891    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzw8\" (UniqueName: \"kubernetes.io/projected/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-kube-api-access-vwzw8\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.758148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.757943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.758148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.757976    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.758148 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:11.758098    2570 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 14 21:58:11.758316 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:11.758173    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert podName:c980cea2-7a18-437e-b8a6-ef5f4bf5f996 nodeName:}" failed. No retries permitted until 2026-07-14 21:58:12.258156445 +0000 UTC m=+432.946132516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-2vm85" (UID: "c980cea2-7a18-437e-b8a6-ef5f4bf5f996") : secret "plugin-serving-cert" not found
Jul 14 21:58:11.758594 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.758576    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:11.781646 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:11.781614    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzw8\" (UniqueName: \"kubernetes.io/projected/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-kube-api-access-vwzw8\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:12.262327 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.262281    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:12.264747 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.264720    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c980cea2-7a18-437e-b8a6-ef5f4bf5f996-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2vm85\" (UID: \"c980cea2-7a18-437e-b8a6-ef5f4bf5f996\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:12.357972 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.357937    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:12.493027 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.492986    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"
Jul 14 21:58:12.631292 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.631255    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:12.754374 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.754338    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85"]
Jul 14 21:58:12.777015 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.776975    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85" event={"ID":"c980cea2-7a18-437e-b8a6-ef5f4bf5f996","Type":"ContainerStarted","Data":"738f530bc54abb3c548a4db57976a7e2759fe103a29447fb0a54e76cb5028642"}
Jul 14 21:58:12.966769 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:12.966734    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:13.082448 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:13.082404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:14.376450 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:14.376407    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:14.468579 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:14.468533    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:16.123193 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:16.123147    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:19.763594 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:19.763556    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-9xn5m"
Jul 14 21:58:19.764120 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:19.763768    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:19.982605 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:19.982564    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:20.180783 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:20.180690    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:20.769815 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:20.769766    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:22.372948 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.372909    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"]
Jul 14 21:58:22.373462 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.373114    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" containerName="manager" containerID="cri-o://2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b" gracePeriod=2
Jul 14 21:58:22.393279 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.393241    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"]
Jul 14 21:58:22.433137 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.433102    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"]
Jul 14 21:58:22.433352 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.433326    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" containerName="manager" containerID="cri-o://db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734" gracePeriod=2
Jul 14 21:58:22.437032 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:22.436973    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:22.462063 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.462015    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"]
Jul 14 21:58:22.463987 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.463843    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:22.464369 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464350    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" containerName="manager"
Jul 14 21:58:22.464452 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464371    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" containerName="manager"
Jul 14 21:58:22.464452 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464392    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" containerName="manager"
Jul 14 21:58:22.464452 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464400    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" containerName="manager"
Jul 14 21:58:22.464611 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464489    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" containerName="manager"
Jul 14 21:58:22.464611 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.464500    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" containerName="manager"
Jul 14 21:58:22.467774 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.467749    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.509126 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.509087    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:22.538164 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.538121    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"]
Jul 14 21:58:22.546110 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.546074    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:22.558164 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.558131    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vxmp\" (UniqueName: \"kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.558374 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.558349    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.571167 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.571125    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"]
Jul 14 21:58:22.575214 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:22.575167    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:22.577844 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:22.577689    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:22.591888 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:22.591840    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:22.637650 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:22.637564    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:22.659163 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.659112    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vxmp\" (UniqueName: \"kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.659353 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.659202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.659353 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.659267    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drss6\" (UniqueName: \"kubernetes.io/projected/92e28a90-22cc-491b-8292-436a447e4c3b-kube-api-access-drss6\") pod \"limitador-operator-controller-manager-859d7b78bd-vx9bm\" (UID: \"92e28a90-22cc-491b-8292-436a447e4c3b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:22.659735 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.659680    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.685237 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.685199    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vxmp\" (UniqueName: \"kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jmr5j\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.760375 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.760332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drss6\" (UniqueName: \"kubernetes.io/projected/92e28a90-22cc-491b-8292-436a447e4c3b-kube-api-access-drss6\") pod \"limitador-operator-controller-manager-859d7b78bd-vx9bm\" (UID: \"92e28a90-22cc-491b-8292-436a447e4c3b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:22.781871 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.781839    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drss6\" (UniqueName: \"kubernetes.io/projected/92e28a90-22cc-491b-8292-436a447e4c3b-kube-api-access-drss6\") pod \"limitador-operator-controller-manager-859d7b78bd-vx9bm\" (UID: \"92e28a90-22cc-491b-8292-436a447e4c3b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:22.801679 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.801639    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:22.858905 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:22.858869    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:24.380839 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:24.380789    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:24.504279 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:24.504235    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:25.067962 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.067917    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 21:58:25.133557 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.133512    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 21:58:25.133739 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.133629    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.284058 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.284019    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.284261 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.284077    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzlz6\" (UniqueName: \"kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.384674 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.384593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.384674 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.384628    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzlz6\" (UniqueName: \"kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.385167 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.385021    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.410617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.410583    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzlz6\" (UniqueName: \"kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6\") pod \"kuadrant-operator-controller-manager-86878c769f-dxjws\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:25.447434 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:25.447397    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:27.475382 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:27.475320    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:27.593659 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:27.593608    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:28.353267 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:28.353228    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:28.587817 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:28.587764    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:31.079240 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:31.079169    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:31.175906 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:31.175867    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:33.970656 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:33.970619    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:35.125063 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.125027    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.128426 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.128373    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.129902 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.129832    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:35.130446 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.130413    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:35.133285 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.133253    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.135960 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.135338    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:35.135960 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:35.135717    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6734db7f_fd8e_4c0d_8fa0_667c9c80f8e8.slice/crio-faa1db5db152b3847d92ce29d241275e239d1b9d0f90f81c0db04fe71cb39665 WatchSource:0}: Error finding container faa1db5db152b3847d92ce29d241275e239d1b9d0f90f81c0db04fe71cb39665: Status 404 returned error can't find the container with id faa1db5db152b3847d92ce29d241275e239d1b9d0f90f81c0db04fe71cb39665
Jul 14 21:58:35.161271 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.161240    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"]
Jul 14 21:58:35.175958 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.175930    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume\") pod \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") "
Jul 14 21:58:35.176147 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.175973    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmfh5\" (UniqueName: \"kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5\") pod \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\" (UID: \"3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed\") "
Jul 14 21:58:35.176147 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.176009    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64h44\" (UniqueName: \"kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44\") pod \"27d63ac9-f826-4dfe-a5d2-36cb78430406\" (UID: \"27d63ac9-f826-4dfe-a5d2-36cb78430406\") "
Jul 14 21:58:35.176506 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.176459    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" (UID: "3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 21:58:35.178538 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:35.178492    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e28a90_22cc_491b_8292_436a447e4c3b.slice/crio-2d9f1e7400754fb4421b7d92672129710f132e538c44919000df9f4421179f31 WatchSource:0}: Error finding container 2d9f1e7400754fb4421b7d92672129710f132e538c44919000df9f4421179f31: Status 404 returned error can't find the container with id 2d9f1e7400754fb4421b7d92672129710f132e538c44919000df9f4421179f31
Jul 14 21:58:35.179337 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.179295    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44" (OuterVolumeSpecName: "kube-api-access-64h44") pod "27d63ac9-f826-4dfe-a5d2-36cb78430406" (UID: "27d63ac9-f826-4dfe-a5d2-36cb78430406"). InnerVolumeSpecName "kube-api-access-64h44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:58:35.179438 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.179376    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5" (OuterVolumeSpecName: "kube-api-access-nmfh5") pod "3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" (UID: "3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed"). InnerVolumeSpecName "kube-api-access-nmfh5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:58:35.180313 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.180234    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 21:58:35.182739 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:58:35.182716    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b75798a_81ca_4d73_b902_d84d89f53d4a.slice/crio-9cb0fbf376cc0d2f00d5d02f55c4311b5cfec32ead1d473b9694cd98601d7b63 WatchSource:0}: Error finding container 9cb0fbf376cc0d2f00d5d02f55c4311b5cfec32ead1d473b9694cd98601d7b63: Status 404 returned error can't find the container with id 9cb0fbf376cc0d2f00d5d02f55c4311b5cfec32ead1d473b9694cd98601d7b63
Jul 14 21:58:35.277525 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.277483    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:58:35.277525 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.277517    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmfh5\" (UniqueName: \"kubernetes.io/projected/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed-kube-api-access-nmfh5\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:58:35.277525 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.277535    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64h44\" (UniqueName: \"kubernetes.io/projected/27d63ac9-f826-4dfe-a5d2-36cb78430406-kube-api-access-64h44\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:58:35.875430 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.875394    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" path="/var/lib/kubelet/pods/27d63ac9-f826-4dfe-a5d2-36cb78430406/volumes"
Jul 14 21:58:35.875759 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.875746    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" path="/var/lib/kubelet/pods/3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed/volumes"
Jul 14 21:58:35.895724 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.895685    2570 generic.go:356] "Generic (PLEG): container finished" podID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" containerID="2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b" exitCode=0
Jul 14 21:58:35.895910 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.895732    2570 scope.go:117] "RemoveContainer" containerID="2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b"
Jul 14 21:58:35.895910 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.895733    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.897352 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.897327    2570 generic.go:356] "Generic (PLEG): container finished" podID="27d63ac9-f826-4dfe-a5d2-36cb78430406" containerID="db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734" exitCode=0
Jul 14 21:58:35.897467 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.897378    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:35.898281 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.898256    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.899923 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.899795    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" event={"ID":"0b75798a-81ca-4d73-b902-d84d89f53d4a","Type":"ContainerStarted","Data":"f66617685757c684d9dd99b0c493d497c06763bb43f316fe21b5c42f53deab83"}
Jul 14 21:58:35.899923 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.899884    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" event={"ID":"0b75798a-81ca-4d73-b902-d84d89f53d4a","Type":"ContainerStarted","Data":"9cb0fbf376cc0d2f00d5d02f55c4311b5cfec32ead1d473b9694cd98601d7b63"}
Jul 14 21:58:35.900103 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.899936    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:35.900359 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.900330    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:35.901592 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.901565    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" event={"ID":"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8","Type":"ContainerStarted","Data":"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"}
Jul 14 21:58:35.901680 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.901620    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" event={"ID":"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8","Type":"ContainerStarted","Data":"faa1db5db152b3847d92ce29d241275e239d1b9d0f90f81c0db04fe71cb39665"}
Jul 14 21:58:35.901680 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.901667    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:35.902098 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.902074    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.903130 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.903109    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm" event={"ID":"92e28a90-22cc-491b-8292-436a447e4c3b","Type":"ContainerStarted","Data":"fbf0107df6357f13d6b6bb996d9eab0c853b1389aa715eedac0d98dc2e6701e1"}
Jul 14 21:58:35.903212 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.903142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm" event={"ID":"92e28a90-22cc-491b-8292-436a447e4c3b","Type":"ContainerStarted","Data":"2d9f1e7400754fb4421b7d92672129710f132e538c44919000df9f4421179f31"}
Jul 14 21:58:35.903212 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.903181    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:35.903994 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.903973    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:35.904724 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.904645    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85" event={"ID":"c980cea2-7a18-437e-b8a6-ef5f4bf5f996","Type":"ContainerStarted","Data":"858322c6560bc3274ebef39430c7c7907511eb12eeba6b9316424a363f8d7a18"}
Jul 14 21:58:35.917350 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.917192    2570 scope.go:117] "RemoveContainer" containerID="2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b"
Jul 14 21:58:35.917534 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.917516    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b\": container with ID starting with 2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b not found: ID does not exist" containerID="2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b"
Jul 14 21:58:35.917581 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.917543    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b"} err="failed to get container status \"2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b\": rpc error: code = NotFound desc = could not find container \"2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b\": container with ID starting with 2408f99cdc78e3b295116b7f9fe38f59e5adacc11d7b13c873bd92b45c921a2b not found: ID does not exist"
Jul 14 21:58:35.917581 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.917562    2570 scope.go:117] "RemoveContainer" containerID="db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734"
Jul 14 21:58:35.924706 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.924654    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" podStartSLOduration=13.924639754 podStartE2EDuration="13.924639754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:35.92341055 +0000 UTC m=+456.611386643" watchObservedRunningTime="2026-07-14 21:58:35.924639754 +0000 UTC m=+456.612615847"
Jul 14 21:58:35.926085 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.926062    2570 scope.go:117] "RemoveContainer" containerID="db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734"
Jul 14 21:58:35.926389 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.926366    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734\": container with ID starting with db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734 not found: ID does not exist" containerID="db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734"
Jul 14 21:58:35.926495 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.926393    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734"} err="failed to get container status \"db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734\": rpc error: code = NotFound desc = could not find container \"db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734\": container with ID starting with db41a0fb5b4f7b11e6d85cd77638af1b83ccb9bcd79a9858eb9007491d461734 not found: ID does not exist"
Jul 14 21:58:35.945820 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.945764    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.946024 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.945987    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" podStartSLOduration=10.945971908 podStartE2EDuration="10.945971908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:35.942959827 +0000 UTC m=+456.630935921" watchObservedRunningTime="2026-07-14 21:58:35.945971908 +0000 UTC m=+456.633947979"
Jul 14 21:58:35.967012 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.966957    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2vm85" podStartSLOduration=2.6608657989999998 podStartE2EDuration="24.966941295s" totalImagesPullingTime="22.306075496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:35.965759108 +0000 UTC m=+456.653735201" watchObservedRunningTime="2026-07-14 21:58:35.966941295 +0000 UTC m=+456.654917388"
Jul 14 21:58:35.968151 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.968118    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-5tz6n\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="3c0a3b16-389b-4df3-bad0-2f45a6e4e1ed" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5tz6n"
Jul 14 21:58:35.988240 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:35.988190    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm" podStartSLOduration=13.988172706 podStartE2EDuration="13.988172706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:58:35.986174373 +0000 UTC m=+456.674150501" watchObservedRunningTime="2026-07-14 21:58:35.988172706 +0000 UTC m=+456.676148798"
Jul 14 21:58:35.990764 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:35.990740    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rjxbx\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="27d63ac9-f826-4dfe-a5d2-36cb78430406" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rjxbx"
Jul 14 21:58:37.623162 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:37.623122    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:37.721323 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:37.721278    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:42.338931 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:42.338898    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:42.443232 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:42.443192    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:45.303537 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:45.303492    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:45.401865 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:45.401833    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:46.114115 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:46.114082    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:46.214553 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:46.214508    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:46.913536 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:46.913498    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:46.913941 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:46.913672    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vx9bm"
Jul 14 21:58:46.913941 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:46.913735    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 21:58:47.015469 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.015431    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:47.015683 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.015648    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" podUID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" containerName="manager" containerID="cri-o://b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926" gracePeriod=10
Jul 14 21:58:47.273105 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.273078    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:47.382880 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.382844    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume\") pod \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") "
Jul 14 21:58:47.383065 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.382946    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vxmp\" (UniqueName: \"kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp\") pod \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\" (UID: \"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8\") "
Jul 14 21:58:47.383236 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.383213    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" (UID: "6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 21:58:47.385065 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.385030    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp" (OuterVolumeSpecName: "kube-api-access-7vxmp") pod "6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" (UID: "6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8"). InnerVolumeSpecName "kube-api-access-7vxmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:58:47.483639 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.483604    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:58:47.483639 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.483635    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vxmp\" (UniqueName: \"kubernetes.io/projected/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8-kube-api-access-7vxmp\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:58:47.969630 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.969589    2570 generic.go:356] "Generic (PLEG): container finished" podID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" containerID="b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926" exitCode=0
Jul 14 21:58:47.970061 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.969640    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"
Jul 14 21:58:47.970061 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.969674    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" event={"ID":"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8","Type":"ContainerDied","Data":"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"}
Jul 14 21:58:47.970061 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.969730    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j" event={"ID":"6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8","Type":"ContainerDied","Data":"faa1db5db152b3847d92ce29d241275e239d1b9d0f90f81c0db04fe71cb39665"}
Jul 14 21:58:47.970061 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.969760    2570 scope.go:117] "RemoveContainer" containerID="b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"
Jul 14 21:58:47.978393 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.978373    2570 scope.go:117] "RemoveContainer" containerID="b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"
Jul 14 21:58:47.978692 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:58:47.978672    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926\": container with ID starting with b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926 not found: ID does not exist" containerID="b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"
Jul 14 21:58:47.978765 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.978703    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926"} err="failed to get container status \"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926\": rpc error: code = NotFound desc = could not find container \"b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926\": container with ID starting with b5583ec323328af7c8c4be437fedf1de535149ec4efb43505f748f1c370b5926 not found: ID does not exist"
Jul 14 21:58:47.992192 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:47.992157    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:48.006446 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:48.006416    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jmr5j"]
Jul 14 21:58:49.879092 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:49.879058    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" path="/var/lib/kubelet/pods/6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8/volumes"
Jul 14 21:58:50.321202 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:50.321165    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:50.437347 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:50.437307    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:56.593172 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:56.593131    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:58:56.688022 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:58:56.687993    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:01.339040 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:01.338996    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:01.443102 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:01.443068    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:03.003526 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:03.003485    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:03.107106 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:03.107067    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:03.474551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:03.474517    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:04.183965 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:04.183925    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:06.393716 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.393682    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:06.394139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.394011    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" containerName="manager"
Jul 14 21:59:06.394139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.394023    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" containerName="manager"
Jul 14 21:59:06.394139 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.394080    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6734db7f-fd8e-4c0d-8fa0-667c9c80f8e8" containerName="manager"
Jul 14 21:59:06.400547 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.400510    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:06.448000 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.447967    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:06.448188 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.448113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.450761 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.450733    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 14 21:59:06.484681 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.484649    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:06.491533 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.491504    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:06.551586 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.551550    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.551753 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.551671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gcm9\" (UniqueName: \"kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.564254 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.564224    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:06.652560 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.652475    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.652560 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.652551    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gcm9\" (UniqueName: \"kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.653368 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.653341    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.665780 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.665740    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gcm9\" (UniqueName: \"kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9\") pod \"limitador-limitador-64c94fd64f-wbrcg\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.760047 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.760003    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:06.923381 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:06.923352    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:06.926132 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:59:06.926100    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534126e3_8d45_44c0_b724_dfeceb37ccf9.slice/crio-79b2ab12f5748cb3809d4aeda359e1cf4bac9b02a460f0aef93b832acf54d6bd WatchSource:0}: Error finding container 79b2ab12f5748cb3809d4aeda359e1cf4bac9b02a460f0aef93b832acf54d6bd: Status 404 returned error can't find the container with id 79b2ab12f5748cb3809d4aeda359e1cf4bac9b02a460f0aef93b832acf54d6bd
Jul 14 21:59:07.057244 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.057204    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" event={"ID":"534126e3-8d45-44c0-b724-dfeceb37ccf9","Type":"ContainerStarted","Data":"79b2ab12f5748cb3809d4aeda359e1cf4bac9b02a460f0aef93b832acf54d6bd"}
Jul 14 21:59:07.196075 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.196039    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:07.271556 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.271515    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:07.288617 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.288572    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:07.431044 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.431009    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:07.431551 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.431155    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:07.433987 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.433967    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6225n\""
Jul 14 21:59:07.561462 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.561368    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp59g\" (UniqueName: \"kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g\") pod \"authorino-75d556bd77-g484v\" (UID: \"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6\") " pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:07.661869 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.661830    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp59g\" (UniqueName: \"kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g\") pod \"authorino-75d556bd77-g484v\" (UID: \"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6\") " pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:07.671330 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.671298    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp59g\" (UniqueName: \"kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g\") pod \"authorino-75d556bd77-g484v\" (UID: \"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6\") " pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:07.741631 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.741593    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:07.950379 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:07.950340    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:07.960460 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:59:07.960426    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e9b5d23_f8d5_48d0_9e95_9a09bba6eef6.slice/crio-dee70b6a36008a55f3a56d67e346da570ca92c9b86bd83ffa8002ecef4477e2c WatchSource:0}: Error finding container dee70b6a36008a55f3a56d67e346da570ca92c9b86bd83ffa8002ecef4477e2c: Status 404 returned error can't find the container with id dee70b6a36008a55f3a56d67e346da570ca92c9b86bd83ffa8002ecef4477e2c
Jul 14 21:59:08.065011 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:08.064972    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g484v" event={"ID":"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6","Type":"ContainerStarted","Data":"dee70b6a36008a55f3a56d67e346da570ca92c9b86bd83ffa8002ecef4477e2c"}
Jul 14 21:59:12.090821 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:12.090778    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" event={"ID":"534126e3-8d45-44c0-b724-dfeceb37ccf9","Type":"ContainerStarted","Data":"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"}
Jul 14 21:59:12.091375 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:12.090856    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:12.092392 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:12.092365    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g484v" event={"ID":"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6","Type":"ContainerStarted","Data":"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"}
Jul 14 21:59:12.107625 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:12.107568    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" podStartSLOduration=1.383287406 podStartE2EDuration="6.107550018s" totalImagesPullingTime="4.724262612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:59:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:59:12.106572058 +0000 UTC m=+492.794548152" watchObservedRunningTime="2026-07-14 21:59:12.107550018 +0000 UTC m=+492.795526111"
Jul 14 21:59:12.121270 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:12.121205    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-g484v" podStartSLOduration=1.430630282 podStartE2EDuration="5.121185369s" totalImagesPullingTime="3.690555087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:59:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:59:12.120863353 +0000 UTC m=+492.808839459" watchObservedRunningTime="2026-07-14 21:59:12.121185369 +0000 UTC m=+492.809161462"
Jul 14 21:59:22.830739 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:22.830702    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:22.831208 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:22.830932    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" podUID="534126e3-8d45-44c0-b724-dfeceb37ccf9" containerName="limitador" containerID="cri-o://0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae" gracePeriod=30
Jul 14 21:59:22.831606 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:22.831583    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:23.383230 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.383202    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:23.499170 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.499135    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gcm9\" (UniqueName: \"kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9\") pod \"534126e3-8d45-44c0-b724-dfeceb37ccf9\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") "
Jul 14 21:59:23.499348 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.499202    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file\") pod \"534126e3-8d45-44c0-b724-dfeceb37ccf9\" (UID: \"534126e3-8d45-44c0-b724-dfeceb37ccf9\") "
Jul 14 21:59:23.499570 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.499549    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file" (OuterVolumeSpecName: "config-file") pod "534126e3-8d45-44c0-b724-dfeceb37ccf9" (UID: "534126e3-8d45-44c0-b724-dfeceb37ccf9"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 21:59:23.501307 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.501285    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9" (OuterVolumeSpecName: "kube-api-access-5gcm9") pod "534126e3-8d45-44c0-b724-dfeceb37ccf9" (UID: "534126e3-8d45-44c0-b724-dfeceb37ccf9"). InnerVolumeSpecName "kube-api-access-5gcm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:59:23.600873 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.600838    2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/534126e3-8d45-44c0-b724-dfeceb37ccf9-config-file\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:59:23.600873 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:23.600865    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gcm9\" (UniqueName: \"kubernetes.io/projected/534126e3-8d45-44c0-b724-dfeceb37ccf9-kube-api-access-5gcm9\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:59:24.150843 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.150723    2570 generic.go:356] "Generic (PLEG): container finished" podID="534126e3-8d45-44c0-b724-dfeceb37ccf9" containerID="0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae" exitCode=0
Jul 14 21:59:24.150843 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.150769    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"
Jul 14 21:59:24.150843 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.150775    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" event={"ID":"534126e3-8d45-44c0-b724-dfeceb37ccf9","Type":"ContainerDied","Data":"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"}
Jul 14 21:59:24.151329 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.150859    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-wbrcg" event={"ID":"534126e3-8d45-44c0-b724-dfeceb37ccf9","Type":"ContainerDied","Data":"79b2ab12f5748cb3809d4aeda359e1cf4bac9b02a460f0aef93b832acf54d6bd"}
Jul 14 21:59:24.151329 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.150890    2570 scope.go:117] "RemoveContainer" containerID="0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"
Jul 14 21:59:24.159553 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.159532    2570 scope.go:117] "RemoveContainer" containerID="0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"
Jul 14 21:59:24.159902 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:59:24.159881    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae\": container with ID starting with 0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae not found: ID does not exist" containerID="0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"
Jul 14 21:59:24.159981 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.159911    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae"} err="failed to get container status \"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae\": rpc error: code = NotFound desc = could not find container \"0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae\": container with ID starting with 0685004fbdce931e8b2483cd3d47911d4c301f57aa64f0d77fc53b5157e58fae not found: ID does not exist"
Jul 14 21:59:24.166920 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.166890    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:24.170795 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:24.170767    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-wbrcg"]
Jul 14 21:59:25.875416 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:25.875386    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="534126e3-8d45-44c0-b724-dfeceb37ccf9" path="/var/lib/kubelet/pods/534126e3-8d45-44c0-b724-dfeceb37ccf9/volumes"
Jul 14 21:59:27.611799 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:27.611761    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:27.712346 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:27.712307    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:28.122027 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.121990    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"]
Jul 14 21:59:28.122343 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.122329    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534126e3-8d45-44c0-b724-dfeceb37ccf9" containerName="limitador"
Jul 14 21:59:28.122399 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.122347    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="534126e3-8d45-44c0-b724-dfeceb37ccf9" containerName="limitador"
Jul 14 21:59:28.122469 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.122456    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="534126e3-8d45-44c0-b724-dfeceb37ccf9" containerName="limitador"
Jul 14 21:59:28.125128 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.125107    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.127753 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.127728    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 14 21:59:28.127986 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.127729    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 21:59:28.127986 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.127870    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-s2p7r\""
Jul 14 21:59:28.128627 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.128613    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 21:59:28.134799 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.134771    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqzn7\" (UniqueName: \"kubernetes.io/projected/a862e6c1-b00a-421d-b88b-a35940940e84-kube-api-access-pqzn7\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.134985 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.134830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a862e6c1-b00a-421d-b88b-a35940940e84-data\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.136097 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.136063    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"]
Jul 14 21:59:28.165614 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.165582    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:28.235439 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.235400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqzn7\" (UniqueName: \"kubernetes.io/projected/a862e6c1-b00a-421d-b88b-a35940940e84-kube-api-access-pqzn7\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.235439 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.235436    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a862e6c1-b00a-421d-b88b-a35940940e84-data\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.235756 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.235741    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a862e6c1-b00a-421d-b88b-a35940940e84-data\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.243618 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.243592    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqzn7\" (UniqueName: \"kubernetes.io/projected/a862e6c1-b00a-421d-b88b-a35940940e84-kube-api-access-pqzn7\") pod \"postgres-69fb4594cf-p6s42\" (UID: \"a862e6c1-b00a-421d-b88b-a35940940e84\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.436234 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.436149    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:28.581528 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:28.581477    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"]
Jul 14 21:59:28.584453 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:59:28.584427    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda862e6c1_b00a_421d_b88b_a35940940e84.slice/crio-14571ec30a4a600b24f39d4f6bf9d10e8990fb913dcaca335cee114d653f5a68 WatchSource:0}: Error finding container 14571ec30a4a600b24f39d4f6bf9d10e8990fb913dcaca335cee114d653f5a68: Status 404 returned error can't find the container with id 14571ec30a4a600b24f39d4f6bf9d10e8990fb913dcaca335cee114d653f5a68
Jul 14 21:59:29.181635 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:29.181602    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42" event={"ID":"a862e6c1-b00a-421d-b88b-a35940940e84","Type":"ContainerStarted","Data":"14571ec30a4a600b24f39d4f6bf9d10e8990fb913dcaca335cee114d653f5a68"}
Jul 14 21:59:34.212053 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:34.212015    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42" event={"ID":"a862e6c1-b00a-421d-b88b-a35940940e84","Type":"ContainerStarted","Data":"88b39ae48f4b088e220a425ebb0289a5ff6eed3ec7e901a5ae9cbe1a1c1e12b7"}
Jul 14 21:59:34.212437 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:34.212098    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:34.234935 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:34.234868    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42" podStartSLOduration=0.998713028 podStartE2EDuration="6.234845436s" totalImagesPullingTime="5.236132408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:59:34.234617497 +0000 UTC m=+514.922593609" watchObservedRunningTime="2026-07-14 21:59:34.234845436 +0000 UTC m=+514.922821531"
Jul 14 21:59:40.247439 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.247407    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6s42"
Jul 14 21:59:40.602874 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.602745    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:40.832392 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.832349    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:40.838878 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.838851    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:40.844620 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.844589    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:40.917534 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.917444    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:40.938624 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:40.938592    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px48r\" (UniqueName: \"kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r\") pod \"authorino-5d4b67c446-vwq5j\" (UID: \"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca\") " pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:41.039427 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.039384    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px48r\" (UniqueName: \"kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r\") pod \"authorino-5d4b67c446-vwq5j\" (UID: \"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca\") " pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:41.050592 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.050555    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-px48r\" (UniqueName: \"kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r\") pod \"authorino-5d4b67c446-vwq5j\" (UID: \"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca\") " pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:41.095900 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.095861    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:41.096236 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.096221    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:41.126919 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.126889    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 21:59:41.132341 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.132312    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.134734 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.134705    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 14 21:59:41.137112 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.137080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 21:59:41.241148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.241114    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpnt\" (UniqueName: \"kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.241148 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.241152    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.245672 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.245643    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:41.248146 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:59:41.248115    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a1f4aa1_e475_4088_96e4_3aa8aafe0aca.slice/crio-545c41181d527ff91ef5c8ef97dab881e3376694ecbde83853c588934649fcb9 WatchSource:0}: Error finding container 545c41181d527ff91ef5c8ef97dab881e3376694ecbde83853c588934649fcb9: Status 404 returned error can't find the container with id 545c41181d527ff91ef5c8ef97dab881e3376694ecbde83853c588934649fcb9
Jul 14 21:59:41.342178 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.342137    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.342358 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.342245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpnt\" (UniqueName: \"kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.344562 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.344541    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.352255 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.352232    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpnt\" (UniqueName: \"kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt\") pod \"authorino-6d645744dc-z8bsb\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") " pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.443943 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.443902    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 21:59:41.579651 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:41.579572    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 21:59:41.581893 ip-10-0-130-163 kubenswrapper[2570]: W0714 21:59:41.581855    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b30c5d7_1855_4675_b9b3_42c926dc8645.slice/crio-40959f13bda665ffb4f4b5a47fe041760ec97f494f9ce907e7e86d8790a9bfb2 WatchSource:0}: Error finding container 40959f13bda665ffb4f4b5a47fe041760ec97f494f9ce907e7e86d8790a9bfb2: Status 404 returned error can't find the container with id 40959f13bda665ffb4f4b5a47fe041760ec97f494f9ce907e7e86d8790a9bfb2
Jul 14 21:59:42.252593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.252485    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d645744dc-z8bsb" event={"ID":"2b30c5d7-1855-4675-b9b3-42c926dc8645","Type":"ContainerStarted","Data":"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"}
Jul 14 21:59:42.252593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.252538    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d645744dc-z8bsb" event={"ID":"2b30c5d7-1855-4675-b9b3-42c926dc8645","Type":"ContainerStarted","Data":"40959f13bda665ffb4f4b5a47fe041760ec97f494f9ce907e7e86d8790a9bfb2"}
Jul 14 21:59:42.254035 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.254012    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" event={"ID":"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca","Type":"ContainerStarted","Data":"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"}
Jul 14 21:59:42.254157 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.254041    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" event={"ID":"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca","Type":"ContainerStarted","Data":"545c41181d527ff91ef5c8ef97dab881e3376694ecbde83853c588934649fcb9"}
Jul 14 21:59:42.254157 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.254063    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" podUID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" containerName="authorino" containerID="cri-o://deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2" gracePeriod=30
Jul 14 21:59:42.270969 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.270913    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d645744dc-z8bsb" podStartSLOduration=0.904298926 podStartE2EDuration="1.270894921s" totalImagesPullingTime="366.595995ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:59:42.269067437 +0000 UTC m=+522.957043530" watchObservedRunningTime="2026-07-14 21:59:42.270894921 +0000 UTC m=+522.958871014"
Jul 14 21:59:42.284246 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.284168    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" podStartSLOduration=1.937400475 podStartE2EDuration="2.284145179s" totalImagesPullingTime="346.744704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 21:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 21:59:42.283492023 +0000 UTC m=+522.971468113" watchObservedRunningTime="2026-07-14 21:59:42.284145179 +0000 UTC m=+522.972121273"
Jul 14 21:59:42.301601 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.301558    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:42.301846 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.301790    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-g484v" podUID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" containerName="authorino" containerID="cri-o://52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384" gracePeriod=30
Jul 14 21:59:42.574912 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.574872    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:42.578242 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.578215    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:42.651693 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.651646    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px48r\" (UniqueName: \"kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r\") pod \"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca\" (UID: \"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca\") "
Jul 14 21:59:42.651693 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.651693    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp59g\" (UniqueName: \"kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g\") pod \"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6\" (UID: \"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6\") "
Jul 14 21:59:42.653977 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.653942    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r" (OuterVolumeSpecName: "kube-api-access-px48r") pod "7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" (UID: "7a1f4aa1-e475-4088-96e4-3aa8aafe0aca"). InnerVolumeSpecName "kube-api-access-px48r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:59:42.654155 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.654138    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g" (OuterVolumeSpecName: "kube-api-access-kp59g") pod "7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" (UID: "7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6"). InnerVolumeSpecName "kube-api-access-kp59g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 21:59:42.753217 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.753176    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px48r\" (UniqueName: \"kubernetes.io/projected/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca-kube-api-access-px48r\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:59:42.753217 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:42.753210    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp59g\" (UniqueName: \"kubernetes.io/projected/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6-kube-api-access-kp59g\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 21:59:43.260114 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.260067    2570 generic.go:356] "Generic (PLEG): container finished" podID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" containerID="52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384" exitCode=0
Jul 14 21:59:43.260512 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.260130    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-g484v"
Jul 14 21:59:43.260512 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.260154    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g484v" event={"ID":"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6","Type":"ContainerDied","Data":"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"}
Jul 14 21:59:43.260512 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.260193    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-g484v" event={"ID":"7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6","Type":"ContainerDied","Data":"dee70b6a36008a55f3a56d67e346da570ca92c9b86bd83ffa8002ecef4477e2c"}
Jul 14 21:59:43.260512 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.260209    2570 scope.go:117] "RemoveContainer" containerID="52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"
Jul 14 21:59:43.261517 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.261494    2570 generic.go:356] "Generic (PLEG): container finished" podID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" containerID="deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2" exitCode=0
Jul 14 21:59:43.261593 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.261578    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vwq5j"
Jul 14 21:59:43.261644 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.261585    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" event={"ID":"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca","Type":"ContainerDied","Data":"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"}
Jul 14 21:59:43.261644 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.261632    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vwq5j" event={"ID":"7a1f4aa1-e475-4088-96e4-3aa8aafe0aca","Type":"ContainerDied","Data":"545c41181d527ff91ef5c8ef97dab881e3376694ecbde83853c588934649fcb9"}
Jul 14 21:59:43.270379 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.270361    2570 scope.go:117] "RemoveContainer" containerID="52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"
Jul 14 21:59:43.270685 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:59:43.270666    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384\": container with ID starting with 52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384 not found: ID does not exist" containerID="52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"
Jul 14 21:59:43.270750 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.270694    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384"} err="failed to get container status \"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384\": rpc error: code = NotFound desc = could not find container \"52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384\": container with ID starting with 52835017d61b76daab3a6420e87a2c7f12b9d63c7c3aa44dbfde68f6581a3384 not found: ID does not exist"
Jul 14 21:59:43.270750 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.270712    2570 scope.go:117] "RemoveContainer" containerID="deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"
Jul 14 21:59:43.278825 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.278791    2570 scope.go:117] "RemoveContainer" containerID="deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"
Jul 14 21:59:43.279117 ip-10-0-130-163 kubenswrapper[2570]: E0714 21:59:43.279095    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2\": container with ID starting with deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2 not found: ID does not exist" containerID="deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"
Jul 14 21:59:43.279162 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.279126    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2"} err="failed to get container status \"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2\": rpc error: code = NotFound desc = could not find container \"deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2\": container with ID starting with deb1cabd2f66e772c34a6ec145469da96549300630b815ec56ae86d4465bcbc2 not found: ID does not exist"
Jul 14 21:59:43.283748 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.283724    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:43.287926 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.287901    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vwq5j"]
Jul 14 21:59:43.297991 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.297966    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:43.302084 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.302063    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-g484v"]
Jul 14 21:59:43.876171 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.876139    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" path="/var/lib/kubelet/pods/7a1f4aa1-e475-4088-96e4-3aa8aafe0aca/volumes"
Jul 14 21:59:43.876483 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:43.876469    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" path="/var/lib/kubelet/pods/7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6/volumes"
Jul 14 21:59:47.088308 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:47.088276    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:47.203630 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:47.203598    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:47.293964 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:47.293929    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:50.383231 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:50.383189    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 21:59:50.484231 ip-10-0-130-163 kubenswrapper[2570]: I0714 21:59:50.484192    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:09.546105 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:09.546068    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:22.180126 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.180076    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:22.283598 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.283547    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:22.699101 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.699065    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:22.731453 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731416    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"]
Jul 14 22:00:22.731793 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731779    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" containerName="authorino"
Jul 14 22:00:22.731882 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731794    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" containerName="authorino"
Jul 14 22:00:22.731882 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731817    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" containerName="authorino"
Jul 14 22:00:22.731882 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731823    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" containerName="authorino"
Jul 14 22:00:22.731882 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731877    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a1f4aa1-e475-4088-96e4-3aa8aafe0aca" containerName="authorino"
Jul 14 22:00:22.732081 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.731886    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e9b5d23-f8d5-48d0-9e95-9a09bba6eef6" containerName="authorino"
Jul 14 22:00:22.734672 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.734655    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:22.737232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.737206    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 14 22:00:22.737232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.737221    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zb4wr\""
Jul 14 22:00:22.745921 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.745898    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"]
Jul 14 22:00:22.905746 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.905702    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr4k8\" (UniqueName: \"kubernetes.io/projected/7ff39070-325b-4447-972c-39af86c78078-kube-api-access-cr4k8\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:22.905746 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.905747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff39070-325b-4447-972c-39af86c78078-openshift-service-ca\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:22.905994 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:22.905789    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.007330 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.007242    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr4k8\" (UniqueName: \"kubernetes.io/projected/7ff39070-325b-4447-972c-39af86c78078-kube-api-access-cr4k8\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.007330 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.007289    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff39070-325b-4447-972c-39af86c78078-openshift-service-ca\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.007535 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.007332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.007535 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:00:23.007426    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 14 22:00:23.008128 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:00:23.007945    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls podName:7ff39070-325b-4447-972c-39af86c78078 nodeName:}" failed. No retries permitted until 2026-07-14 22:00:23.507914964 +0000 UTC m=+564.195891049 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls") pod "maas-api-8559b688dd-sh5pc" (UID: "7ff39070-325b-4447-972c-39af86c78078") : secret "maas-api-serving-cert" not found
Jul 14 22:00:23.008500 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.008475    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ff39070-325b-4447-972c-39af86c78078-openshift-service-ca\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.017762 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.017727    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr4k8\" (UniqueName: \"kubernetes.io/projected/7ff39070-325b-4447-972c-39af86c78078-kube-api-access-cr4k8\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.261403 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.261309    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"]
Jul 14 22:00:23.264832 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.264798    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.267272 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.267248    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mwv4h\""
Jul 14 22:00:23.267409 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.267374    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 14 22:00:23.276534 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.276505    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"]
Jul 14 22:00:23.410221 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.410178    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsfqf\" (UniqueName: \"kubernetes.io/projected/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-kube-api-access-tsfqf\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.410419 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.410270    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.427086 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.427046    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:23.510943 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.510904    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.511105 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.510963    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsfqf\" (UniqueName: \"kubernetes.io/projected/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-kube-api-access-tsfqf\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.511105 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.511036    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.511850 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.511769    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.513998 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.513968    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ff39070-325b-4447-972c-39af86c78078-maas-api-tls\") pod \"maas-api-8559b688dd-sh5pc\" (UID: \"7ff39070-325b-4447-972c-39af86c78078\") " pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.520598 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.520570    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsfqf\" (UniqueName: \"kubernetes.io/projected/294cb6a0-fcdf-4a18-b372-7cf2f48ba629-kube-api-access-tsfqf\") pod \"payload-pre-processing-74584bb9f8-kcssg\" (UID: \"294cb6a0-fcdf-4a18-b372-7cf2f48ba629\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.524464 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.524434    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:23.577556 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.577515    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:23.645956 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.645923    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:23.728574 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.728547    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"]
Jul 14 22:00:23.731748 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:00:23.731722    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294cb6a0_fcdf_4a18_b372_7cf2f48ba629.slice/crio-c199dab5a8cc117bb41d2a73fbba1d4aabb3dd070aab4cac6aa457d05dbb2f4a WatchSource:0}: Error finding container c199dab5a8cc117bb41d2a73fbba1d4aabb3dd070aab4cac6aa457d05dbb2f4a: Status 404 returned error can't find the container with id c199dab5a8cc117bb41d2a73fbba1d4aabb3dd070aab4cac6aa457d05dbb2f4a
Jul 14 22:00:23.793400 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:23.793371    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"]
Jul 14 22:00:23.795772 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:00:23.795749    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ff39070_325b_4447_972c_39af86c78078.slice/crio-fa9e205c7470ed9a036149ba15c8d018241e0df6896a6717f3239d9823f30c6b WatchSource:0}: Error finding container fa9e205c7470ed9a036149ba15c8d018241e0df6896a6717f3239d9823f30c6b: Status 404 returned error can't find the container with id fa9e205c7470ed9a036149ba15c8d018241e0df6896a6717f3239d9823f30c6b
Jul 14 22:00:24.461784 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:24.461688    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg" event={"ID":"294cb6a0-fcdf-4a18-b372-7cf2f48ba629","Type":"ContainerStarted","Data":"c199dab5a8cc117bb41d2a73fbba1d4aabb3dd070aab4cac6aa457d05dbb2f4a"}
Jul 14 22:00:24.463364 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:24.463329    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc" event={"ID":"7ff39070-325b-4447-972c-39af86c78078","Type":"ContainerStarted","Data":"fa9e205c7470ed9a036149ba15c8d018241e0df6896a6717f3239d9823f30c6b"}
Jul 14 22:00:26.475735 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.475699    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc" event={"ID":"7ff39070-325b-4447-972c-39af86c78078","Type":"ContainerStarted","Data":"01514229c393979d31d24b0e85c3be6ac573ac4554f5b0feec721637471c4ab2"}
Jul 14 22:00:26.476233 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.475768    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:26.477160 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.477139    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg" event={"ID":"294cb6a0-fcdf-4a18-b372-7cf2f48ba629","Type":"ContainerStarted","Data":"2bcdc0587787f7c3c6bee1325bcfb4ac720dfb908dce2b3296d81dfea8c45775"}
Jul 14 22:00:26.477277 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.477256    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:26.500186 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.500077    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc" podStartSLOduration=2.006953212 podStartE2EDuration="4.500059526s" totalImagesPullingTime="2.493106314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:00:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:00:26.496944219 +0000 UTC m=+567.184920312" watchObservedRunningTime="2026-07-14 22:00:26.500059526 +0000 UTC m=+567.188035619"
Jul 14 22:00:26.518167 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.518115    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg" podStartSLOduration=1.009433269 podStartE2EDuration="3.518098927s" totalImagesPullingTime="2.508665658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:00:26.517725075 +0000 UTC m=+567.205701167" watchObservedRunningTime="2026-07-14 22:00:26.518098927 +0000 UTC m=+567.206075020"
Jul 14 22:00:26.774066 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.773980    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:26.874842 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:26.874794    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:32.487125 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:32.487094    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8559b688dd-sh5pc"
Jul 14 22:00:33.792488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:33.792452    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:34.743992 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:34.743957    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:37.483983 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:37.483955    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-kcssg"
Jul 14 22:00:52.482704 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:52.482670    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:59.719845 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:59.719794    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:00:59.821559 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:59.821531    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:00:59.823360 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:59.823340    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:00:59.830065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:59.830037    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:00:59.831850 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:00:59.831816    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:01:01.142138 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.142105    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:01.147629 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.147597    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"]
Jul 14 22:01:01.149931 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.149905    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.153528 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.153506    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9zfdw\""
Jul 14 22:01:01.153686 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.153540    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 22:01:01.153686 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.153550    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 14 22:01:01.159034 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.159011    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 22:01:01.166232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.166206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"]
Jul 14 22:01:01.245713 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245678    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.245929 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245727    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.245929 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245750    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.245929 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245795    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.245929 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m25gj\" (UniqueName: \"kubernetes.io/projected/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-kube-api-access-m25gj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.245929 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.245894    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347098 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347060    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347303 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347115    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m25gj\" (UniqueName: \"kubernetes.io/projected/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-kube-api-access-m25gj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347303 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347303 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347331    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347359    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347613 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347591    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347658 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347645    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.347738 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.347720    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.349509 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.349489    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.349678 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.349663    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.359779 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.359755    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m25gj\" (UniqueName: \"kubernetes.io/projected/8b09fa8a-809e-4d5d-8e0b-f6233f19077b-kube-api-access-m25gj\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hlm4c\" (UID: \"8b09fa8a-809e-4d5d-8e0b-f6233f19077b\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.462691 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.462652    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:01.605402 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.605375    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"]
Jul 14 22:01:01.607423 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:01:01.607399    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b09fa8a_809e_4d5d_8e0b_f6233f19077b.slice/crio-4c6beedf4f0ba7a068ec216d408f6bdc9209b285d0d34206d40ac3a35b68b5d9 WatchSource:0}: Error finding container 4c6beedf4f0ba7a068ec216d408f6bdc9209b285d0d34206d40ac3a35b68b5d9: Status 404 returned error can't find the container with id 4c6beedf4f0ba7a068ec216d408f6bdc9209b285d0d34206d40ac3a35b68b5d9
Jul 14 22:01:01.608843 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.608828    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:01:01.645503 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:01.645467    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c" event={"ID":"8b09fa8a-809e-4d5d-8e0b-f6233f19077b","Type":"ContainerStarted","Data":"4c6beedf4f0ba7a068ec216d408f6bdc9209b285d0d34206d40ac3a35b68b5d9"}
Jul 14 22:01:03.658532 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:03.658494    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c" event={"ID":"8b09fa8a-809e-4d5d-8e0b-f6233f19077b","Type":"ContainerStarted","Data":"e9ccdaaad5a37d375cb833c13fc7d2c7437caaa61d3ab61065011149961a6198"}
Jul 14 22:01:03.685335 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:03.685118    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c" podStartSLOduration=1.6047382049999999 podStartE2EDuration="2.685092784s" totalImagesPullingTime="1.080354579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:01:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:01:03.676427868 +0000 UTC m=+604.364403974" watchObservedRunningTime="2026-07-14 22:01:03.685092784 +0000 UTC m=+604.373068878"
Jul 14 22:01:11.463580 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:11.463540    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:11.464051 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:11.463636    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:11.476226 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:11.476200    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:11.707421 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:11.707391    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hlm4c"
Jul 14 22:01:11.859479 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:11.859397    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:22.298505 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:22.298405    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:41.183795 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.183753    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"]
Jul 14 22:01:41.187773 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.187747    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.190831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.190772    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 14 22:01:41.199182 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.199147    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"]
Jul 14 22:01:41.226320 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.226276    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:41.307554 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307516    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.307777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307577    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.307777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307646    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.307777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307715    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876b5a27-e6cf-43e0-b701-8289bf394abc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.307777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307755    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.307777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.307773    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7sj\" (UniqueName: \"kubernetes.io/projected/876b5a27-e6cf-43e0-b701-8289bf394abc-kube-api-access-cf7sj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.408673 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408635    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.408673 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408679    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.408977 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408709    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876b5a27-e6cf-43e0-b701-8289bf394abc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.408977 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408921    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.408977 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408941    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7sj\" (UniqueName: \"kubernetes.io/projected/876b5a27-e6cf-43e0-b701-8289bf394abc-kube-api-access-cf7sj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.409150 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.408992    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.409150 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.409129    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.409248 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.409168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.409290 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.409256    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.411243 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.411215    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/876b5a27-e6cf-43e0-b701-8289bf394abc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.411514 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.411492    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/876b5a27-e6cf-43e0-b701-8289bf394abc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.418873 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.418844    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7sj\" (UniqueName: \"kubernetes.io/projected/876b5a27-e6cf-43e0-b701-8289bf394abc-kube-api-access-cf7sj\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq\" (UID: \"876b5a27-e6cf-43e0-b701-8289bf394abc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.499526 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.499485    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:41.652320 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.652282    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"]
Jul 14 22:01:41.659361 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:01:41.656731    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876b5a27_e6cf_43e0_b701_8289bf394abc.slice/crio-11f7da934a0968de050bbf3b2d2b89b8f565548a91c5e8a20516e562094784e0 WatchSource:0}: Error finding container 11f7da934a0968de050bbf3b2d2b89b8f565548a91c5e8a20516e562094784e0: Status 404 returned error can't find the container with id 11f7da934a0968de050bbf3b2d2b89b8f565548a91c5e8a20516e562094784e0
Jul 14 22:01:41.836573 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.836475    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq" event={"ID":"876b5a27-e6cf-43e0-b701-8289bf394abc","Type":"ContainerStarted","Data":"11f7da934a0968de050bbf3b2d2b89b8f565548a91c5e8a20516e562094784e0"}
Jul 14 22:01:41.902172 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:41.902139    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:42.001704 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:42.001651    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:42.843432 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:42.843390    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq" event={"ID":"876b5a27-e6cf-43e0-b701-8289bf394abc","Type":"ContainerStarted","Data":"a675aaed67c959eb2bef9856252d84208eb4c749d2e8e19a9a4f45857b0cabc5"}
Jul 14 22:01:42.861978 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:42.861926    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq" podStartSLOduration=1.662534005 podStartE2EDuration="1.86190856s" totalImagesPullingTime="199.374555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:01:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:01:42.859831912 +0000 UTC m=+643.547807999" watchObservedRunningTime="2026-07-14 22:01:42.86190856 +0000 UTC m=+643.549884653"
Jul 14 22:01:51.500019 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:51.499980    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:51.500433 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:51.500047    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:51.512853 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:51.512800    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:51.900235 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:51.900144    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq"
Jul 14 22:01:58.902058 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.902024    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"]
Jul 14 22:01:58.904486 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.904457    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.907100 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.907077    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 14 22:01:58.916222 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.916191    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"]
Jul 14 22:01:58.945133 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.945078    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:01:58.979199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979155    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.979199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.979409 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979229    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.979409 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979299    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j59st\" (UniqueName: \"kubernetes.io/projected/09fcb1d8-8e11-414a-bd1b-4701754a3b38-kube-api-access-j59st\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.979409 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979404    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:58.979506 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:58.979425    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j59st\" (UniqueName: \"kubernetes.io/projected/09fcb1d8-8e11-414a-bd1b-4701754a3b38-kube-api-access-j59st\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080546    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080572    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080619    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.080720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.080710    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.081090 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.081064    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.081162 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.081103    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.081162 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.081072    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.083172 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.083146    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09fcb1d8-8e11-414a-bd1b-4701754a3b38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.083344 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.083325    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09fcb1d8-8e11-414a-bd1b-4701754a3b38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.090308 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.090280    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j59st\" (UniqueName: \"kubernetes.io/projected/09fcb1d8-8e11-414a-bd1b-4701754a3b38-kube-api-access-j59st\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-pp687\" (UID: \"09fcb1d8-8e11-414a-bd1b-4701754a3b38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.216495 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.216450    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:01:59.363479 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.363323    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"]
Jul 14 22:01:59.365783 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:01:59.365752    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09fcb1d8_8e11_414a_bd1b_4701754a3b38.slice/crio-ecf386f31e55d7d92b11472d69eb318eff01e9789d852f5c17c57e37438cc3c0 WatchSource:0}: Error finding container ecf386f31e55d7d92b11472d69eb318eff01e9789d852f5c17c57e37438cc3c0: Status 404 returned error can't find the container with id ecf386f31e55d7d92b11472d69eb318eff01e9789d852f5c17c57e37438cc3c0
Jul 14 22:01:59.930720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.930627    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687" event={"ID":"09fcb1d8-8e11-414a-bd1b-4701754a3b38","Type":"ContainerStarted","Data":"71775466750ee0bd9e017c2f01c19df9e72b7afbb9409e02d32c9498050604f1"}
Jul 14 22:01:59.930720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.930680    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687" event={"ID":"09fcb1d8-8e11-414a-bd1b-4701754a3b38","Type":"ContainerStarted","Data":"ecf386f31e55d7d92b11472d69eb318eff01e9789d852f5c17c57e37438cc3c0"}
Jul 14 22:01:59.949046 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:01:59.948984    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687" podStartSLOduration=1.767157605 podStartE2EDuration="1.94896228s" totalImagesPullingTime="181.804675ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:01:59.946854266 +0000 UTC m=+660.634830361" watchObservedRunningTime="2026-07-14 22:01:59.94896228 +0000 UTC m=+660.636938374"
Jul 14 22:02:09.217521 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:02:09.217479    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:02:09.218120 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:02:09.217581    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:02:09.231355 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:02:09.231317    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:02:09.994357 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:02:09.994326    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-pp687"
Jul 14 22:02:10.230428 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:02:10.230392    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:03:01.091563 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:01.091471    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:03:01.190379 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:01.190339    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:03:01.883465 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:01.883431    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 22:03:01.883677 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:01.883648    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6d645744dc-z8bsb" podUID="2b30c5d7-1855-4675-b9b3-42c926dc8645" containerName="authorino" containerID="cri-o://aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd" gracePeriod=30
Jul 14 22:03:02.150149 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.150118    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 22:03:02.232189 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.232134    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:03:02.247699 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.247661    2570 generic.go:356] "Generic (PLEG): container finished" podID="2b30c5d7-1855-4675-b9b3-42c926dc8645" containerID="aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd" exitCode=0
Jul 14 22:03:02.247892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.247709    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d645744dc-z8bsb" event={"ID":"2b30c5d7-1855-4675-b9b3-42c926dc8645","Type":"ContainerDied","Data":"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"}
Jul 14 22:03:02.247892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.247736    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d645744dc-z8bsb" event={"ID":"2b30c5d7-1855-4675-b9b3-42c926dc8645","Type":"ContainerDied","Data":"40959f13bda665ffb4f4b5a47fe041760ec97f494f9ce907e7e86d8790a9bfb2"}
Jul 14 22:03:02.247892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.247752    2570 scope.go:117] "RemoveContainer" containerID="aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"
Jul 14 22:03:02.247892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.247710    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d645744dc-z8bsb"
Jul 14 22:03:02.259816 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.259770    2570 scope.go:117] "RemoveContainer" containerID="aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"
Jul 14 22:03:02.260218 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:03:02.260188    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd\": container with ID starting with aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd not found: ID does not exist" containerID="aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"
Jul 14 22:03:02.260293 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.260222    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd"} err="failed to get container status \"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd\": rpc error: code = NotFound desc = could not find container \"aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd\": container with ID starting with aed24eec07436600c51ccfd954cdaa85edf0ba27c3e0c4b0a1bfcf5982660cbd not found: ID does not exist"
Jul 14 22:03:02.272579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.272543    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnpnt\" (UniqueName: \"kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt\") pod \"2b30c5d7-1855-4675-b9b3-42c926dc8645\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") "
Jul 14 22:03:02.272768 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.272611    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert\") pod \"2b30c5d7-1855-4675-b9b3-42c926dc8645\" (UID: \"2b30c5d7-1855-4675-b9b3-42c926dc8645\") "
Jul 14 22:03:02.274791 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.274760    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt" (OuterVolumeSpecName: "kube-api-access-xnpnt") pod "2b30c5d7-1855-4675-b9b3-42c926dc8645" (UID: "2b30c5d7-1855-4675-b9b3-42c926dc8645"). InnerVolumeSpecName "kube-api-access-xnpnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:03:02.286182 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.286126    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2b30c5d7-1855-4675-b9b3-42c926dc8645" (UID: "2b30c5d7-1855-4675-b9b3-42c926dc8645"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:03:02.327439 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.327390    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:03:02.373799 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.373766    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnpnt\" (UniqueName: \"kubernetes.io/projected/2b30c5d7-1855-4675-b9b3-42c926dc8645-kube-api-access-xnpnt\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:03:02.373799 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.373794    2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b30c5d7-1855-4675-b9b3-42c926dc8645-tls-cert\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:03:02.570038 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.570004    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 22:03:02.571627 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:02.571605    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d645744dc-z8bsb"]
Jul 14 22:03:03.875944 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:03.875912    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b30c5d7-1855-4675-b9b3-42c926dc8645" path="/var/lib/kubelet/pods/2b30c5d7-1855-4675-b9b3-42c926dc8645/volumes"
Jul 14 22:03:43.986244 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:03:43.986202    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:04:12.916354 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:04:12.916313    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:04:14.888517 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:04:14.888475    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:04:57.713618 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:04:57.713578    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:04:57.814141 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:04:57.814101    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:04:57.927654 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:04:57.927612    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:03.609541 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:03.609499    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:04.496282 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:04.496241    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:04.597787 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:04.597744    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:10.528797 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:10.528752    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:10.624454 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:10.624420    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:53.513463 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:53.513379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:05:59.852220 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:59.852183    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:05:59.855791 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:59.855752    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:05:59.860062 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:59.860037    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:05:59.863217 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:05:59.863192    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:06:03.328219 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:06:03.328184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:07:25.154153 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:07:25.154062    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:23.903158 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:23.903119    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 22:09:23.903616 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:23.903360    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" podUID="0b75798a-81ca-4d73-b902-d84d89f53d4a" containerName="manager" containerID="cri-o://f66617685757c684d9dd99b0c493d497c06763bb43f316fe21b5c42f53deab83" gracePeriod=10
Jul 14 22:09:27.150673 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.150638    2570 generic.go:356] "Generic (PLEG): container finished" podID="0b75798a-81ca-4d73-b902-d84d89f53d4a" containerID="f66617685757c684d9dd99b0c493d497c06763bb43f316fe21b5c42f53deab83" exitCode=0
Jul 14 22:09:27.151101 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.150693    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" event={"ID":"0b75798a-81ca-4d73-b902-d84d89f53d4a","Type":"ContainerDied","Data":"f66617685757c684d9dd99b0c493d497c06763bb43f316fe21b5c42f53deab83"}
Jul 14 22:09:27.666294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.666257    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 22:09:27.832336 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.832239    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume\") pod \"0b75798a-81ca-4d73-b902-d84d89f53d4a\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") "
Jul 14 22:09:27.832491 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.832369    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzlz6\" (UniqueName: \"kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6\") pod \"0b75798a-81ca-4d73-b902-d84d89f53d4a\" (UID: \"0b75798a-81ca-4d73-b902-d84d89f53d4a\") "
Jul 14 22:09:27.832619 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.832595    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b75798a-81ca-4d73-b902-d84d89f53d4a" (UID: "0b75798a-81ca-4d73-b902-d84d89f53d4a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:09:27.834534 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.834508    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6" (OuterVolumeSpecName: "kube-api-access-fzlz6") pod "0b75798a-81ca-4d73-b902-d84d89f53d4a" (UID: "0b75798a-81ca-4d73-b902-d84d89f53d4a"). InnerVolumeSpecName "kube-api-access-fzlz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:09:27.933254 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.933214    2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b75798a-81ca-4d73-b902-d84d89f53d4a-extensions-socket-volume\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:09:27.933254 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:27.933248    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzlz6\" (UniqueName: \"kubernetes.io/projected/0b75798a-81ca-4d73-b902-d84d89f53d4a-kube-api-access-fzlz6\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:09:28.160104 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:28.160008    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"
Jul 14 22:09:28.160104 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:28.160024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws" event={"ID":"0b75798a-81ca-4d73-b902-d84d89f53d4a","Type":"ContainerDied","Data":"9cb0fbf376cc0d2f00d5d02f55c4311b5cfec32ead1d473b9694cd98601d7b63"}
Jul 14 22:09:28.160104 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:28.160083    2570 scope.go:117] "RemoveContainer" containerID="f66617685757c684d9dd99b0c493d497c06763bb43f316fe21b5c42f53deab83"
Jul 14 22:09:28.177638 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:28.177606    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 22:09:28.179287 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:28.179265    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dxjws"]
Jul 14 22:09:29.876913 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:29.876873    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b75798a-81ca-4d73-b902-d84d89f53d4a" path="/var/lib/kubelet/pods/0b75798a-81ca-4d73-b902-d84d89f53d4a/volumes"
Jul 14 22:09:30.423070 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:30.423032    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:09:30.512084 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:09:30.512050    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:10:18.621743 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:18.621697    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:10:25.096372 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096330    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"]
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096701    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b30c5d7-1855-4675-b9b3-42c926dc8645" containerName="authorino"
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096711    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b30c5d7-1855-4675-b9b3-42c926dc8645" containerName="authorino"
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096741    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b75798a-81ca-4d73-b902-d84d89f53d4a" containerName="manager"
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096747    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b75798a-81ca-4d73-b902-d84d89f53d4a" containerName="manager"
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096816    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b30c5d7-1855-4675-b9b3-42c926dc8645" containerName="authorino"
Jul 14 22:10:25.096875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.096830    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b75798a-81ca-4d73-b902-d84d89f53d4a" containerName="manager"
Jul 14 22:10:25.100971 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.100946    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.103878 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.103847    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5pgl8\""
Jul 14 22:10:25.111393 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.111358    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"]
Jul 14 22:10:25.149720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.149671    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40df9a6b-14e3-4e89-a223-02388737a0cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.149946 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.149860    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4vqz\" (UniqueName: \"kubernetes.io/projected/40df9a6b-14e3-4e89-a223-02388737a0cd-kube-api-access-p4vqz\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.250617 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.250576    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4vqz\" (UniqueName: \"kubernetes.io/projected/40df9a6b-14e3-4e89-a223-02388737a0cd-kube-api-access-p4vqz\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.250819 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.250634    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40df9a6b-14e3-4e89-a223-02388737a0cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.251156 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.251129    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40df9a6b-14e3-4e89-a223-02388737a0cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.259069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.259035    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4vqz\" (UniqueName: \"kubernetes.io/projected/40df9a6b-14e3-4e89-a223-02388737a0cd-kube-api-access-p4vqz\") pod \"kuadrant-operator-controller-manager-86878c769f-c6q94\" (UID: \"40df9a6b-14e3-4e89-a223-02388737a0cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.413081 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.412983    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:25.566687 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:25.566659    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"]
Jul 14 22:10:25.568829 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:10:25.568784    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40df9a6b_14e3_4e89_a223_02388737a0cd.slice/crio-4c944aa7a1ab26e3c00666151223670b3430ff9a35abc8eb50b152dda14dd4d2 WatchSource:0}: Error finding container 4c944aa7a1ab26e3c00666151223670b3430ff9a35abc8eb50b152dda14dd4d2: Status 404 returned error can't find the container with id 4c944aa7a1ab26e3c00666151223670b3430ff9a35abc8eb50b152dda14dd4d2
Jul 14 22:10:26.444267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:26.444231    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94" event={"ID":"40df9a6b-14e3-4e89-a223-02388737a0cd","Type":"ContainerStarted","Data":"e818b2f4e2ed3d3cdd75dca7f894c1122e8423affa02a6745e36322752d5f0b9"}
Jul 14 22:10:26.444688 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:26.444282    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94" event={"ID":"40df9a6b-14e3-4e89-a223-02388737a0cd","Type":"ContainerStarted","Data":"4c944aa7a1ab26e3c00666151223670b3430ff9a35abc8eb50b152dda14dd4d2"}
Jul 14 22:10:26.444688 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:26.444372    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:26.469357 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:26.469299    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94" podStartSLOduration=1.469281622 podStartE2EDuration="1.469281622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:10:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:10:26.467130256 +0000 UTC m=+1167.155106392" watchObservedRunningTime="2026-07-14 22:10:26.469281622 +0000 UTC m=+1167.157257715"
Jul 14 22:10:37.452313 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:37.452278    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c6q94"
Jul 14 22:10:59.881064 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:59.881028    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:10:59.885226 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:59.885200    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:10:59.889121 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:59.889091    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:10:59.892981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:10:59.892955    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:13:33.608067 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:13:33.607975    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:13:35.025657 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:13:35.025622    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:14:55.663518 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:14:55.663431    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:15:59.910850 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:15:59.910799    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:15:59.914370 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:15:59.914344    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:15:59.921366 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:15:59.921335    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:15:59.925301 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:15:59.925275    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:16:23.200201 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:23.200157    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:29.317844 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:29.317777    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:34.209945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:34.209913    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:36.815530 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:36.815500    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:45.204374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:45.204336    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:50.486264 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:50.486226    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:53.198979 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:53.198941    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:53.298937 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:53.298904    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:53.677626 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:53.677546    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:59.631339 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:59.631302    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:59.729065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:59.729023    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:59.856873 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:59.856837    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:16:59.956924 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:16:59.956888    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:00.939915 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:00.939876    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:01.041096 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.041056    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:01.250098 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.250010    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:17:01.253266 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.253237    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.255996 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.255975    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5574436c-serving-cert\""
Jul 14 22:17:01.278724 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.278688    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:17:01.335564 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.335528    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:01.347094 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.347067    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc54h\" (UniqueName: \"kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.347273 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.347117    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.347273 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.347140    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.428971 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.428934    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:01.448349 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.448315    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.448522 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.448389    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc54h\" (UniqueName: \"kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.448522 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.448434    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.449008 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.448987    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.450840 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.450799    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.461092 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.461064    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc54h\" (UniqueName: \"kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h\") pod \"maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.567301 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.567207    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:01.711681 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.711652    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:17:01.714396 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:17:01.714353    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80ac5592_7c54_4541_b510_c708b306438f.slice/crio-f3c5f5e8e39d97a3e9d3dbe40a75e199e7ebe461edb948a9e6550ac41aaaf05f WatchSource:0}: Error finding container f3c5f5e8e39d97a3e9d3dbe40a75e199e7ebe461edb948a9e6550ac41aaaf05f: Status 404 returned error can't find the container with id f3c5f5e8e39d97a3e9d3dbe40a75e199e7ebe461edb948a9e6550ac41aaaf05f
Jul 14 22:17:01.715829 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.715783    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:17:01.814188 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.814125    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:01.914917 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:01.914777    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:02.369726 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:02.369688    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" event={"ID":"80ac5592-7c54-4541-b510-c708b306438f","Type":"ContainerStarted","Data":"f3c5f5e8e39d97a3e9d3dbe40a75e199e7ebe461edb948a9e6550ac41aaaf05f"}
Jul 14 22:17:03.375944 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:03.375904    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" event={"ID":"80ac5592-7c54-4541-b510-c708b306438f","Type":"ContainerStarted","Data":"22fe6c939416053db96018ae22c7d9f8a6d9bf2f830f50df13a89972703982a0"}
Jul 14 22:17:03.376431 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:03.376019    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:17:03.394946 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:03.394893    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" podStartSLOduration=1.5837417999999999 podStartE2EDuration="2.394873495s" totalImagesPullingTime="811.131695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:17:03.393539397 +0000 UTC m=+1564.081515494" watchObservedRunningTime="2026-07-14 22:17:03.394873495 +0000 UTC m=+1564.082849588"
Jul 14 22:17:03.663924 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:03.663888    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:17:09.386164 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:17:09.386133    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:18:16.486762 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.486695    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:18:16.487419 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.487021    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" podUID="80ac5592-7c54-4541-b510-c708b306438f" containerName="maas-api" containerID="cri-o://22fe6c939416053db96018ae22c7d9f8a6d9bf2f830f50df13a89972703982a0" gracePeriod=30
Jul 14 22:18:16.533555 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.533523    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:16.727878 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.727847    2570 generic.go:356] "Generic (PLEG): container finished" podID="80ac5592-7c54-4541-b510-c708b306438f" containerID="22fe6c939416053db96018ae22c7d9f8a6d9bf2f830f50df13a89972703982a0" exitCode=0
Jul 14 22:18:16.728018 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.727906    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" event={"ID":"80ac5592-7c54-4541-b510-c708b306438f","Type":"ContainerDied","Data":"22fe6c939416053db96018ae22c7d9f8a6d9bf2f830f50df13a89972703982a0"}
Jul 14 22:18:16.748300 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.748222    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:18:16.832673 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.832633    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca\") pod \"80ac5592-7c54-4541-b510-c708b306438f\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") "
Jul 14 22:18:16.832875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.832760    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wc54h\" (UniqueName: \"kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h\") pod \"80ac5592-7c54-4541-b510-c708b306438f\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") "
Jul 14 22:18:16.832875 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.832850    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls\") pod \"80ac5592-7c54-4541-b510-c708b306438f\" (UID: \"80ac5592-7c54-4541-b510-c708b306438f\") "
Jul 14 22:18:16.833115 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.833075    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80ac5592-7c54-4541-b510-c708b306438f" (UID: "80ac5592-7c54-4541-b510-c708b306438f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:18:16.835214 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.835182    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h" (OuterVolumeSpecName: "kube-api-access-wc54h") pod "80ac5592-7c54-4541-b510-c708b306438f" (UID: "80ac5592-7c54-4541-b510-c708b306438f"). InnerVolumeSpecName "kube-api-access-wc54h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:18:16.835214 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.835196    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80ac5592-7c54-4541-b510-c708b306438f" (UID: "80ac5592-7c54-4541-b510-c708b306438f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:18:16.933878 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.933840    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wc54h\" (UniqueName: \"kubernetes.io/projected/80ac5592-7c54-4541-b510-c708b306438f-kube-api-access-wc54h\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:18:16.933878 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.933874    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80ac5592-7c54-4541-b510-c708b306438f-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:18:16.933878 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:16.933884    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80ac5592-7c54-4541-b510-c708b306438f-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:18:17.734857 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.734826    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm" event={"ID":"80ac5592-7c54-4541-b510-c708b306438f","Type":"ContainerDied","Data":"f3c5f5e8e39d97a3e9d3dbe40a75e199e7ebe461edb948a9e6550ac41aaaf05f"}
Jul 14 22:18:17.734857 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.734852    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"
Jul 14 22:18:17.735369 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.734888    2570 scope.go:117] "RemoveContainer" containerID="22fe6c939416053db96018ae22c7d9f8a6d9bf2f830f50df13a89972703982a0"
Jul 14 22:18:17.757533 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.757490    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:18:17.763230 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.763197    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5574436c-79bcbfc98d-6ghkm"]
Jul 14 22:18:17.876726 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:17.876687    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80ac5592-7c54-4541-b510-c708b306438f" path="/var/lib/kubelet/pods/80ac5592-7c54-4541-b510-c708b306438f/volumes"
Jul 14 22:18:21.132126 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:21.132081    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:27.252271 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:27.252231    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:27.324759 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:27.324727    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:31.427656 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:31.427618    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:31.703411 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:31.703369    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:31.794658 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:31.794625    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:31.901603 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:31.901573    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:33.124924 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.124893    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:33.236709 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.236672    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:18:33.237230 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.237208    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80ac5592-7c54-4541-b510-c708b306438f" containerName="maas-api"
Jul 14 22:18:33.237309 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.237233    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="80ac5592-7c54-4541-b510-c708b306438f" containerName="maas-api"
Jul 14 22:18:33.237390 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.237366    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="80ac5592-7c54-4541-b510-c708b306438f" containerName="maas-api"
Jul 14 22:18:33.246205 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.246165    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.250904 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.249818    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-de92a977-serving-cert\""
Jul 14 22:18:33.251653 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.251626    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:18:33.279404 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.279367    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:33.380038 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.379944    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn4hw\" (UniqueName: \"kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.380038 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.380030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.380252 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.380071    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.389205 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.389177    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:33.481579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.481538    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.481579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.481579    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn4hw\" (UniqueName: \"kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.481824 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.481643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.482372 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.482348    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.484640 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.484604    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.494449 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.494405    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn4hw\" (UniqueName: \"kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw\") pod \"maas-api-e2e-migrate-de92a977-86c467978b-wfksw\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.559701 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.559659    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:33.724559 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.724520    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:18:33.726991 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:18:33.726954    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd510015c_99e5_46a0_a470_b833d8dc61c0.slice/crio-29eda920cb8489c65e176af1558d877d0e96ca2a4d878731dd8b9d113b4f45ea WatchSource:0}: Error finding container 29eda920cb8489c65e176af1558d877d0e96ca2a4d878731dd8b9d113b4f45ea: Status 404 returned error can't find the container with id 29eda920cb8489c65e176af1558d877d0e96ca2a4d878731dd8b9d113b4f45ea
Jul 14 22:18:33.818426 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.818393    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" event={"ID":"d510015c-99e5-46a0-a470-b833d8dc61c0","Type":"ContainerStarted","Data":"29eda920cb8489c65e176af1558d877d0e96ca2a4d878731dd8b9d113b4f45ea"}
Jul 14 22:18:33.925525 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:33.925426    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:34.087037 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:34.087003    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:34.825216 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:34.825181    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" event={"ID":"d510015c-99e5-46a0-a470-b833d8dc61c0","Type":"ContainerStarted","Data":"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"}
Jul 14 22:18:34.825607 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:34.825253    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:34.844313 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:34.844268    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" podStartSLOduration=1.460153142 podStartE2EDuration="1.844249508s" totalImagesPullingTime="384.096366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:18:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:18:34.842972941 +0000 UTC m=+1655.530949033" watchObservedRunningTime="2026-07-14 22:18:34.844249508 +0000 UTC m=+1655.532225601"
Jul 14 22:18:36.785919 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:36.785884    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:38.219604 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:38.219569    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:18:40.836592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:40.836561    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:18:41.203135 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:18:41.203107    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:37.213445 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:37.213353    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:37.317780 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:37.317743    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:37.870601 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:37.870562    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:38.049037 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:38.048995    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:38.354666 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:38.354629    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:38.455566 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:38.455531    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:43.697509 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:43.697469    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:19:43.697982 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:43.697740    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" podUID="d510015c-99e5-46a0-a470-b833d8dc61c0" containerName="maas-api" containerID="cri-o://083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7" gracePeriod=30
Jul 14 22:19:43.723610 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:43.723578    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:43.962773 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:43.962687    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:19:43.982984 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:43.980872    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:44.108213 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.108150    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca\") pod \"d510015c-99e5-46a0-a470-b833d8dc61c0\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") "
Jul 14 22:19:44.108435 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.108266    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mn4hw\" (UniqueName: \"kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw\") pod \"d510015c-99e5-46a0-a470-b833d8dc61c0\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") "
Jul 14 22:19:44.108435 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.108335    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls\") pod \"d510015c-99e5-46a0-a470-b833d8dc61c0\" (UID: \"d510015c-99e5-46a0-a470-b833d8dc61c0\") "
Jul 14 22:19:44.108634 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.108608    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d510015c-99e5-46a0-a470-b833d8dc61c0" (UID: "d510015c-99e5-46a0-a470-b833d8dc61c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:19:44.110518 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.110475    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d510015c-99e5-46a0-a470-b833d8dc61c0" (UID: "d510015c-99e5-46a0-a470-b833d8dc61c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:19:44.110632 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.110534    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw" (OuterVolumeSpecName: "kube-api-access-mn4hw") pod "d510015c-99e5-46a0-a470-b833d8dc61c0" (UID: "d510015c-99e5-46a0-a470-b833d8dc61c0"). InnerVolumeSpecName "kube-api-access-mn4hw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:19:44.159487 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.159448    2570 generic.go:356] "Generic (PLEG): container finished" podID="d510015c-99e5-46a0-a470-b833d8dc61c0" containerID="083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7" exitCode=0
Jul 14 22:19:44.159663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.159508    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"
Jul 14 22:19:44.159663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.159526    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" event={"ID":"d510015c-99e5-46a0-a470-b833d8dc61c0","Type":"ContainerDied","Data":"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"}
Jul 14 22:19:44.159663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.159602    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw" event={"ID":"d510015c-99e5-46a0-a470-b833d8dc61c0","Type":"ContainerDied","Data":"29eda920cb8489c65e176af1558d877d0e96ca2a4d878731dd8b9d113b4f45ea"}
Jul 14 22:19:44.159663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.159625    2570 scope.go:117] "RemoveContainer" containerID="083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"
Jul 14 22:19:44.171479 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.171428    2570 scope.go:117] "RemoveContainer" containerID="083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"
Jul 14 22:19:44.171949 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:19:44.171916    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7\": container with ID starting with 083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7 not found: ID does not exist" containerID="083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"
Jul 14 22:19:44.172045 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.171960    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7"} err="failed to get container status \"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7\": rpc error: code = NotFound desc = could not find container \"083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7\": container with ID starting with 083170e87e21f3051f613ef350b7c32a0534316026418cedb8be467bac09e9e7 not found: ID does not exist"
Jul 14 22:19:44.181935 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.181899    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:19:44.187699 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.187664    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-de92a977-86c467978b-wfksw"]
Jul 14 22:19:44.209920 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.209887    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn4hw\" (UniqueName: \"kubernetes.io/projected/d510015c-99e5-46a0-a470-b833d8dc61c0-kube-api-access-mn4hw\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:19:44.209920 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.209917    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d510015c-99e5-46a0-a470-b833d8dc61c0-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:19:44.209920 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.209930    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d510015c-99e5-46a0-a470-b833d8dc61c0-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:19:44.363561 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:44.363504    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:45.875902 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:45.875856    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d510015c-99e5-46a0-a470-b833d8dc61c0" path="/var/lib/kubelet/pods/d510015c-99e5-46a0-a470-b833d8dc61c0/volumes"
Jul 14 22:19:49.822473 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:49.822433    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:49.898989 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:49.898943    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:50.071866 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:50.071793    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:50.169947 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:50.169863    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:51.179187 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.179153    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:51.253440 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.253404    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:19:51.253827 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.253797    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d510015c-99e5-46a0-a470-b833d8dc61c0" containerName="maas-api"
Jul 14 22:19:51.253904 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.253830    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d510015c-99e5-46a0-a470-b833d8dc61c0" containerName="maas-api"
Jul 14 22:19:51.253965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.253906    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d510015c-99e5-46a0-a470-b833d8dc61c0" containerName="maas-api"
Jul 14 22:19:51.258246 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.258219    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.261096 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.261070    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c29be32a-serving-cert\""
Jul 14 22:19:51.269287 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.269260    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:19:51.270903 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.270854    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.271069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.270920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gpnj\" (UniqueName: \"kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.271069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.271021    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.272064 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.272034    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:51.346509 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.346470    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:51.372525 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.372494    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.372730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.372532    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gpnj\" (UniqueName: \"kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.372730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.372605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.372909 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:19:51.372728    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-serving-cert: secret "maas-api-e2e-ait-c29be32a-serving-cert" not found
Jul 14 22:19:51.372909 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:19:51.372827    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls podName:b20bc8ea-70e2-433f-b1b2-78911601aa1b nodeName:}" failed. No retries permitted until 2026-07-14 22:19:51.872787373 +0000 UTC m=+1732.560763458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls") pod "maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" (UID: "b20bc8ea-70e2-433f-b1b2-78911601aa1b") : secret "maas-api-e2e-ait-c29be32a-serving-cert" not found
Jul 14 22:19:51.373333 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.373311    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.390432 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.390400    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gpnj\" (UniqueName: \"kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.853973 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.853935    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:51.877553 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.877510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:51.877739 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:19:51.877656    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-serving-cert: secret "maas-api-e2e-ait-c29be32a-serving-cert" not found
Jul 14 22:19:51.877739 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:19:51.877729    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls podName:b20bc8ea-70e2-433f-b1b2-78911601aa1b nodeName:}" failed. No retries permitted until 2026-07-14 22:19:52.877711491 +0000 UTC m=+1733.565687561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls") pod "maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" (UID: "b20bc8ea-70e2-433f-b1b2-78911601aa1b") : secret "maas-api-e2e-ait-c29be32a-serving-cert" not found
Jul 14 22:19:51.956411 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:51.956379    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:52.054436 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:52.054404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:52.885345 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:52.885310    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:52.887772 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:52.887746    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:53.071973 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:53.071924    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:53.222010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:53.221981    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:19:53.224296 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:19:53.224263    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20bc8ea_70e2_433f_b1b2_78911601aa1b.slice/crio-9279160947cbc6c54233f4ef312d10cc4e2ab1cf18d5a7719008b7d716150b8e WatchSource:0}: Error finding container 9279160947cbc6c54233f4ef312d10cc4e2ab1cf18d5a7719008b7d716150b8e: Status 404 returned error can't find the container with id 9279160947cbc6c54233f4ef312d10cc4e2ab1cf18d5a7719008b7d716150b8e
Jul 14 22:19:53.506101 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:53.506027    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:19:54.210541 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:54.210442    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" event={"ID":"b20bc8ea-70e2-433f-b1b2-78911601aa1b","Type":"ContainerStarted","Data":"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"}
Jul 14 22:19:54.210541 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:54.210503    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:19:54.210541 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:54.210523    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" event={"ID":"b20bc8ea-70e2-433f-b1b2-78911601aa1b","Type":"ContainerStarted","Data":"9279160947cbc6c54233f4ef312d10cc4e2ab1cf18d5a7719008b7d716150b8e"}
Jul 14 22:19:54.235219 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:19:54.235152    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" podStartSLOduration=2.503779667 podStartE2EDuration="3.235127167s" totalImagesPullingTime="731.3475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:19:54.232858015 +0000 UTC m=+1734.920834145" watchObservedRunningTime="2026-07-14 22:19:54.235127167 +0000 UTC m=+1734.923103260"
Jul 14 22:20:00.222340 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:00.222257    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:20:54.816353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:54.816240    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:20:56.763216 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:56.761949    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:20:56.763216 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:56.762291    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" podUID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" containerName="maas-api" containerID="cri-o://16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8" gracePeriod=30
Jul 14 22:20:56.788402 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:56.788340    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:20:57.026155 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.026081    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:20:57.176446 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.176412    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca\") pod \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") "
Jul 14 22:20:57.176652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.176472    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gpnj\" (UniqueName: \"kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj\") pod \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") "
Jul 14 22:20:57.176652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.176601    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") pod \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\" (UID: \"b20bc8ea-70e2-433f-b1b2-78911601aa1b\") "
Jul 14 22:20:57.176912 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.176875    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b20bc8ea-70e2-433f-b1b2-78911601aa1b" (UID: "b20bc8ea-70e2-433f-b1b2-78911601aa1b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:20:57.177008 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.176958    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b20bc8ea-70e2-433f-b1b2-78911601aa1b-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:20:57.178787 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.178756    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b20bc8ea-70e2-433f-b1b2-78911601aa1b" (UID: "b20bc8ea-70e2-433f-b1b2-78911601aa1b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:20:57.179037 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.179012    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj" (OuterVolumeSpecName: "kube-api-access-9gpnj") pod "b20bc8ea-70e2-433f-b1b2-78911601aa1b" (UID: "b20bc8ea-70e2-433f-b1b2-78911601aa1b"). InnerVolumeSpecName "kube-api-access-9gpnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:20:57.277884 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.277757    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b20bc8ea-70e2-433f-b1b2-78911601aa1b-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:20:57.277884 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.277789    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gpnj\" (UniqueName: \"kubernetes.io/projected/b20bc8ea-70e2-433f-b1b2-78911601aa1b-kube-api-access-9gpnj\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:20:57.529300 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.529211    2570 generic.go:356] "Generic (PLEG): container finished" podID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" containerID="16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8" exitCode=0
Jul 14 22:20:57.529462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.529318    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" event={"ID":"b20bc8ea-70e2-433f-b1b2-78911601aa1b","Type":"ContainerDied","Data":"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"}
Jul 14 22:20:57.529462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.529335    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"
Jul 14 22:20:57.529462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.529367    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb" event={"ID":"b20bc8ea-70e2-433f-b1b2-78911601aa1b","Type":"ContainerDied","Data":"9279160947cbc6c54233f4ef312d10cc4e2ab1cf18d5a7719008b7d716150b8e"}
Jul 14 22:20:57.529462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.529395    2570 scope.go:117] "RemoveContainer" containerID="16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"
Jul 14 22:20:57.540179 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.540152    2570 scope.go:117] "RemoveContainer" containerID="16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"
Jul 14 22:20:57.540548 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:20:57.540526    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8\": container with ID starting with 16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8 not found: ID does not exist" containerID="16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"
Jul 14 22:20:57.540622 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.540556    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8"} err="failed to get container status \"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8\": rpc error: code = NotFound desc = could not find container \"16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8\": container with ID starting with 16807099134b8ca07cb280d6c6ae4c7b80b54905e057a65fa7d5d157da78aeb8 not found: ID does not exist"
Jul 14 22:20:57.550664 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.550631    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:20:57.554501 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.554472    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c29be32a-867b48b44d-ndgpb"]
Jul 14 22:20:57.875815 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:57.875718    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" path="/var/lib/kubelet/pods/b20bc8ea-70e2-433f-b1b2-78911601aa1b/volumes"
Jul 14 22:20:59.942869 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:59.942838    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:20:59.949203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:59.949172    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:20:59.950738 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:59.950711    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:20:59.956752 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:20:59.956729    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:21:01.428326 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:01.428291    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:07.282050 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:07.282014    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:07.382840 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:07.382770    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:12.454310 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:12.454271    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:12.776182 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:12.776145    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:13.971726 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:13.971689    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.056248 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.056189    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:21:14.056685 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.056664    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" containerName="maas-api"
Jul 14 22:21:14.056685 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.056687    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" containerName="maas-api"
Jul 14 22:21:14.056879 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.056868    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b20bc8ea-70e2-433f-b1b2-78911601aa1b" containerName="maas-api"
Jul 14 22:21:14.070321 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.070282    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.072444 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.072413    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.073382 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.073255    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:21:14.077072 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.077044    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-6b2eb522-serving-cert\""
Jul 14 22:21:14.215518 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.215481    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.233101 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.232996    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clzf9\" (UniqueName: \"kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.233101 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.233064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.233344 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.233169    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.334439 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.334383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clzf9\" (UniqueName: \"kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.334647 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.334456    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.334647 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.334518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.334647 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:21:14.334604    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-serving-cert: secret "maas-api-e2e-derive-6b2eb522-serving-cert" not found
Jul 14 22:21:14.334845 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:21:14.334684    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls podName:e57ac07e-2f0a-45a8-be5b-4fa2049427c1 nodeName:}" failed. No retries permitted until 2026-07-14 22:21:14.834663786 +0000 UTC m=+1815.522639861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls") pod "maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" (UID: "e57ac07e-2f0a-45a8-be5b-4fa2049427c1") : secret "maas-api-e2e-derive-6b2eb522-serving-cert" not found
Jul 14 22:21:14.335355 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.335333    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.370486 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.370452    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clzf9\" (UniqueName: \"kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.647566 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.647475    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.839831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.839764    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.842448 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.842409    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") pod \"maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:14.844098 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.844070    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.944852 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.944799    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:14.986302 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:14.986266    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:15.146602 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:15.146557    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:21:15.146908 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:21:15.146879    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode57ac07e_2f0a_45a8_be5b_4fa2049427c1.slice/crio-3a1204706ba02f7bf4522c8433ad90abd2c752f8c8cadc12dfdf8ecc581606cc WatchSource:0}: Error finding container 3a1204706ba02f7bf4522c8433ad90abd2c752f8c8cadc12dfdf8ecc581606cc: Status 404 returned error can't find the container with id 3a1204706ba02f7bf4522c8433ad90abd2c752f8c8cadc12dfdf8ecc581606cc
Jul 14 22:21:15.618892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:15.618839    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" event={"ID":"e57ac07e-2f0a-45a8-be5b-4fa2049427c1","Type":"ContainerStarted","Data":"9682d0a20ad86107b7d87a73e658fd8be651c7f243968c38f11106511c722713"}
Jul 14 22:21:15.619095 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:15.618915    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:15.619095 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:15.618939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" event={"ID":"e57ac07e-2f0a-45a8-be5b-4fa2049427c1","Type":"ContainerStarted","Data":"3a1204706ba02f7bf4522c8433ad90abd2c752f8c8cadc12dfdf8ecc581606cc"}
Jul 14 22:21:15.637894 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:15.637836    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" podStartSLOduration=1.2944131890000001 podStartE2EDuration="1.637783665s" totalImagesPullingTime="343.370476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:21:15.635945157 +0000 UTC m=+1816.323921254" watchObservedRunningTime="2026-07-14 22:21:15.637783665 +0000 UTC m=+1816.325759757"
Jul 14 22:21:19.264404 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:19.264371    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:19.366456 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:19.366420    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:21:21.629845 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:21.629782    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:21:22.634000 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:21:22.633966    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:23.140292 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:23.140191    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:28.757737 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:28.757694    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:22:28.758182 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:28.757995    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" podUID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" containerName="maas-api" containerID="cri-o://9682d0a20ad86107b7d87a73e658fd8be651c7f243968c38f11106511c722713" gracePeriod=30
Jul 14 22:22:28.799394 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:28.799354    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:28.975394 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:28.975363    2570 generic.go:356] "Generic (PLEG): container finished" podID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" containerID="9682d0a20ad86107b7d87a73e658fd8be651c7f243968c38f11106511c722713" exitCode=0
Jul 14 22:22:28.975574 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:28.975431    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" event={"ID":"e57ac07e-2f0a-45a8-be5b-4fa2049427c1","Type":"ContainerDied","Data":"9682d0a20ad86107b7d87a73e658fd8be651c7f243968c38f11106511c722713"}
Jul 14 22:22:29.026695 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.026668    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:22:29.086858 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.086792    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clzf9\" (UniqueName: \"kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9\") pod \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") "
Jul 14 22:22:29.086858 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.086853    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca\") pod \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") "
Jul 14 22:22:29.087119 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.086888    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") pod \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\" (UID: \"e57ac07e-2f0a-45a8-be5b-4fa2049427c1\") "
Jul 14 22:22:29.087230 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.087204    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e57ac07e-2f0a-45a8-be5b-4fa2049427c1" (UID: "e57ac07e-2f0a-45a8-be5b-4fa2049427c1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:22:29.088994 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.088969    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e57ac07e-2f0a-45a8-be5b-4fa2049427c1" (UID: "e57ac07e-2f0a-45a8-be5b-4fa2049427c1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:22:29.089089 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.089065    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9" (OuterVolumeSpecName: "kube-api-access-clzf9") pod "e57ac07e-2f0a-45a8-be5b-4fa2049427c1" (UID: "e57ac07e-2f0a-45a8-be5b-4fa2049427c1"). InnerVolumeSpecName "kube-api-access-clzf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:22:29.187945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.187906    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clzf9\" (UniqueName: \"kubernetes.io/projected/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-kube-api-access-clzf9\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:22:29.187945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.187938    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:22:29.187945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.187948    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e57ac07e-2f0a-45a8-be5b-4fa2049427c1-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:22:29.982173 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.982136    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"
Jul 14 22:22:29.982646 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.982138    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk" event={"ID":"e57ac07e-2f0a-45a8-be5b-4fa2049427c1","Type":"ContainerDied","Data":"3a1204706ba02f7bf4522c8433ad90abd2c752f8c8cadc12dfdf8ecc581606cc"}
Jul 14 22:22:29.982646 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.982272    2570 scope.go:117] "RemoveContainer" containerID="9682d0a20ad86107b7d87a73e658fd8be651c7f243968c38f11106511c722713"
Jul 14 22:22:29.999057 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:29.999026    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:22:30.002601 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:30.002574    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-6b2eb522-557dd4db6d-vw7nk"]
Jul 14 22:22:31.880460 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:31.880414    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" path="/var/lib/kubelet/pods/e57ac07e-2f0a-45a8-be5b-4fa2049427c1/volumes"
Jul 14 22:22:33.411615 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:33.411561    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:39.649302 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:39.649255    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:39.748165 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:39.748133    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:43.611001 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:43.610962    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:44.233017 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:44.232975    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:44.326833 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:44.326770    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:44.427254 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:44.427218    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:45.976412 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:45.976375    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:50.808720 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:50.808657    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:51.823100 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:51.823045    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:51.886304 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:51.886258    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:51.980483 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:51.980428    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:52.078228 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:52.078121    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:52.178340 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:52.178291    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:22:52.561985 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:22:52.561951    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:18.582959 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:18.582921    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:24.461005 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:24.460969    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:24.523751 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:24.523713    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:24.617877 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:24.617792    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:24.711479 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:24.711361    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:40.562702 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:40.562663    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:46.370742 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:46.370688    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:46.432466 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:46.432429    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:46.535556 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:46.535507    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:46.633910 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:46.633816    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:57.153059 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:57.152972    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:23:57.972557 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:23:57.972516    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:03.822518 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:03.822480    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:03.923593 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:03.923555    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:04.022850 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:04.022814    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:04.122909 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:04.122796    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:05.019474 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:05.019441    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:05.156972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:05.156924    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:06.905602 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:06.905565    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:12.767965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:12.767928    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:13.042385 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:13.042293    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:13.527689 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:13.527655    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:19.671398 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:19.671362    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:19.761925 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:19.761892    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:19.891625 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:19.891586    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:20.205143 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:20.205106    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:20.644354 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:20.644265    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:23.468610 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.468570    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:23.563361 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.563319    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:23.663159 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.663121    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:23.663507 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.663495    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" containerName="maas-api"
Jul 14 22:24:23.663556 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.663509    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" containerName="maas-api"
Jul 14 22:24:23.663602 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.663581    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57ac07e-2f0a-45a8-be5b-4fa2049427c1" containerName="maas-api"
Jul 14 22:24:23.666690 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.666662    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.669178 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.669148    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c488720f-serving-cert\""
Jul 14 22:24:23.677333 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.677139    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:23.696758 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.696722    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:23.819428 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.819327    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr95v\" (UniqueName: \"kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.819608 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.819442    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.819608 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.819486    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.920336 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.920291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.920537 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.920451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.920592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.920534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr95v\" (UniqueName: \"kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.920646 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:24:23.920600    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-serving-cert: secret "maas-api-e2e-mt-c488720f-serving-cert" not found
Jul 14 22:24:23.920698 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:24:23.920673    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls podName:cec94a26-965a-41b8-8c65-e4d176b5a093 nodeName:}" failed. No retries permitted until 2026-07-14 22:24:24.420651191 +0000 UTC m=+2005.108627276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls") pod "maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" (UID: "cec94a26-965a-41b8-8c65-e4d176b5a093") : secret "maas-api-e2e-mt-c488720f-serving-cert" not found
Jul 14 22:24:23.921039 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.921021    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:23.929748 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:23.929716    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr95v\" (UniqueName: \"kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:24.182992 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.182890    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:24.282411 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.282375    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:24.382794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.382756    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:24.425911 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.425880    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:24.428466 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.428438    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") pod \"maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:24.582272 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.582230    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:24.732314 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.732275    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:24.734272 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:24:24.734239    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcec94a26_965a_41b8_8c65_e4d176b5a093.slice/crio-2b4c7a548d465279db3a54f3a616d405d11d5e0705ec9bbf884180e3cd0fd313 WatchSource:0}: Error finding container 2b4c7a548d465279db3a54f3a616d405d11d5e0705ec9bbf884180e3cd0fd313: Status 404 returned error can't find the container with id 2b4c7a548d465279db3a54f3a616d405d11d5e0705ec9bbf884180e3cd0fd313
Jul 14 22:24:24.736011 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:24.735989    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:24:25.552421 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:25.552388    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" event={"ID":"cec94a26-965a-41b8-8c65-e4d176b5a093","Type":"ContainerStarted","Data":"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"}
Jul 14 22:24:25.552639 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:25.552577    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:25.552639 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:25.552614    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" event={"ID":"cec94a26-965a-41b8-8c65-e4d176b5a093","Type":"ContainerStarted","Data":"2b4c7a548d465279db3a54f3a616d405d11d5e0705ec9bbf884180e3cd0fd313"}
Jul 14 22:24:25.569503 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:25.569451    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" podStartSLOduration=1.963635714 podStartE2EDuration="2.569432123s" totalImagesPullingTime="605.796409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:24:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:24:25.568276769 +0000 UTC m=+2006.256252879" watchObservedRunningTime="2026-07-14 22:24:25.569432123 +0000 UTC m=+2006.257408216"
Jul 14 22:24:27.997065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:27.997026    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:28.702555 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:28.702522    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:28.797439 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:28.797407    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:29.140549 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:29.140453    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:31.563972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:31.563941    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:35.217741 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:35.217700    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:35.283358 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:35.283316    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:38.154327 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:38.154271    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:38.255511 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:38.255475    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:46.145378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.145342    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:46.245589 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.245533    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:46.390409 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.390376    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:46.390934 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.390884    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerName="maas-api" containerID="cri-o://863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d" gracePeriod=30
Jul 14 22:24:46.420020 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.419990    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:46.558544 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.558506    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.47:8443/health\": dial tcp 10.133.0.47:8443: connect: connection refused"
Jul 14 22:24:46.656679 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.656611    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:46.657981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.657956    2570 generic.go:356] "Generic (PLEG): container finished" podID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerID="863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d" exitCode=0
Jul 14 22:24:46.658102 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.658024    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" event={"ID":"cec94a26-965a-41b8-8c65-e4d176b5a093","Type":"ContainerDied","Data":"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"}
Jul 14 22:24:46.658102 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.658077    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j" event={"ID":"cec94a26-965a-41b8-8c65-e4d176b5a093","Type":"ContainerDied","Data":"2b4c7a548d465279db3a54f3a616d405d11d5e0705ec9bbf884180e3cd0fd313"}
Jul 14 22:24:46.658102 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.658099    2570 scope.go:117] "RemoveContainer" containerID="863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"
Jul 14 22:24:46.666690 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.666669    2570 scope.go:117] "RemoveContainer" containerID="863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"
Jul 14 22:24:46.667055 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:24:46.667036    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d\": container with ID starting with 863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d not found: ID does not exist" containerID="863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"
Jul 14 22:24:46.667140 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.667065    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d"} err="failed to get container status \"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d\": rpc error: code = NotFound desc = could not find container \"863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d\": container with ID starting with 863b50960f39d57886eb43b658be1b12e1ce729aa72791f788ce612aae8fa83d not found: ID does not exist"
Jul 14 22:24:46.730738 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.730688    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr95v\" (UniqueName: \"kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v\") pod \"cec94a26-965a-41b8-8c65-e4d176b5a093\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") "
Jul 14 22:24:46.730946 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.730785    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca\") pod \"cec94a26-965a-41b8-8c65-e4d176b5a093\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") "
Jul 14 22:24:46.730946 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.730877    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") pod \"cec94a26-965a-41b8-8c65-e4d176b5a093\" (UID: \"cec94a26-965a-41b8-8c65-e4d176b5a093\") "
Jul 14 22:24:46.731507 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.731475    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cec94a26-965a-41b8-8c65-e4d176b5a093" (UID: "cec94a26-965a-41b8-8c65-e4d176b5a093"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:24:46.733488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.733462    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cec94a26-965a-41b8-8c65-e4d176b5a093" (UID: "cec94a26-965a-41b8-8c65-e4d176b5a093"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:24:46.733488 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.733476    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v" (OuterVolumeSpecName: "kube-api-access-mr95v") pod "cec94a26-965a-41b8-8c65-e4d176b5a093" (UID: "cec94a26-965a-41b8-8c65-e4d176b5a093"). InnerVolumeSpecName "kube-api-access-mr95v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:24:46.831981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.831945    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr95v\" (UniqueName: \"kubernetes.io/projected/cec94a26-965a-41b8-8c65-e4d176b5a093-kube-api-access-mr95v\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:24:46.831981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.831973    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cec94a26-965a-41b8-8c65-e4d176b5a093-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:24:46.831981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:46.831984    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cec94a26-965a-41b8-8c65-e4d176b5a093-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:24:47.664169 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:47.664141    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"
Jul 14 22:24:47.684656 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:47.684623    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:47.688263 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:47.688237    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c488720f-5d5984bb98-wcc2j"]
Jul 14 22:24:47.875263 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:47.875229    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" path="/var/lib/kubelet/pods/cec94a26-965a-41b8-8c65-e4d176b5a093/volumes"
Jul 14 22:24:50.705928 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:50.705893    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:56.455048 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:56.455017    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:24:56.554787 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:24:56.554757    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:01.399275 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:01.399238    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:01.634218 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:01.634184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:01.811910 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:01.811865    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:01.884616 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:01.884578    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:02.100651 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:02.100545    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:07.675894 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:07.675852    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:07.743753 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:07.743719    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:07.840341 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:07.840287    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:07.940347 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:07.940315    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:08.063181 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:08.063145    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:13.774699 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:13.774664    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:13.879476 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:13.879440    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:20.704583 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:20.704548    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:21.016603 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:21.016485    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:21.185554 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:21.185497    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:23.770461 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.770412    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:23.922765 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.922725    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:25:23.923256 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.923236    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerName="maas-api"
Jul 14 22:25:23.923256 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.923258    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerName="maas-api"
Jul 14 22:25:23.923402 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.923345    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cec94a26-965a-41b8-8c65-e4d176b5a093" containerName="maas-api"
Jul 14 22:25:23.926447 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.926389    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:23.929078 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.929049    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ba078dbe-serving-cert\""
Jul 14 22:25:23.951038 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.950994    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:25:23.973711 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.973665    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:23.973917 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.973731    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:23.973917 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:23.973870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdbxs\" (UniqueName: \"kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.075042 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.074926    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.075042 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.075004    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.075276 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.075070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdbxs\" (UniqueName: \"kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.075276 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:25:24.075093    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-serving-cert: secret "maas-api-e2e-mt-ba078dbe-serving-cert" not found
Jul 14 22:25:24.075276 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:25:24.075167    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls podName:e12396de-a551-4e03-8077-c4c2c6e76094 nodeName:}" failed. No retries permitted until 2026-07-14 22:25:24.575149415 +0000 UTC m=+2065.263125485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls") pod "maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" (UID: "e12396de-a551-4e03-8077-c4c2c6e76094") : secret "maas-api-e2e-mt-ba078dbe-serving-cert" not found
Jul 14 22:25:24.075880 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.075857    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.081308 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.081279    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:24.090577 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.090547    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdbxs\" (UniqueName: \"kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.180244 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.180205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:24.494393 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.494358    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:24.580199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.580152    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.582663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.582642    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") pod \"maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.592159 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.592129    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:24.837373 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.837261    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:24.982847 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:24.982821    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:25:24.985320 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:25:24.985278    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12396de_a551_4e03_8077_c4c2c6e76094.slice/crio-8b7b270f138866410083e4103f0bed9c73907cd56cb243ed3cebd0671b9ab9a9 WatchSource:0}: Error finding container 8b7b270f138866410083e4103f0bed9c73907cd56cb243ed3cebd0671b9ab9a9: Status 404 returned error can't find the container with id 8b7b270f138866410083e4103f0bed9c73907cd56cb243ed3cebd0671b9ab9a9
Jul 14 22:25:25.850886 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:25.850844    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" event={"ID":"e12396de-a551-4e03-8077-c4c2c6e76094","Type":"ContainerStarted","Data":"8b7b270f138866410083e4103f0bed9c73907cd56cb243ed3cebd0671b9ab9a9"}
Jul 14 22:25:26.857520 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:26.857484    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" event={"ID":"e12396de-a551-4e03-8077-c4c2c6e76094","Type":"ContainerStarted","Data":"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"}
Jul 14 22:25:26.857966 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:26.857562    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:26.875069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:26.875016    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" podStartSLOduration=2.819994737 podStartE2EDuration="3.874997352s" totalImagesPullingTime="1.055002615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:25:26.87292903 +0000 UTC m=+2067.560905141" watchObservedRunningTime="2026-07-14 22:25:26.874997352 +0000 UTC m=+2067.562973444"
Jul 14 22:25:28.284860 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:28.284832    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:32.868152 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:32.868114    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:25:41.273843 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:41.273787    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:41.374212 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:41.374184    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:46.298306 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.298268    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:25:46.301848 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.301799    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.304796 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.304770    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ba078dbe\"/\"openshift-service-ca.crt\""
Jul 14 22:25:46.305970 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.305949    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ba078dbe\"/\"default-dockercfg-6gvmc\""
Jul 14 22:25:46.306081 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.305947    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ba078dbe\"/\"e2e-lifecycle-model-ba078dbe-kserve-self-signed-certs\""
Jul 14 22:25:46.306081 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.306006    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ba078dbe\"/\"kube-root-ca.crt\""
Jul 14 22:25:46.321132 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.321099    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:25:46.326634 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.326591    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:25:46.380241 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380194    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.380441 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380323    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.380441 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380387    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.380528 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380443    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.380528 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380466    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.380528 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.380483    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48df8\" (UniqueName: \"kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481577 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481761 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481601    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481761 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481624    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481761 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481640    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48df8\" (UniqueName: \"kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481761 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.481761 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.481699    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.482077 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.482046    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.482077 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.482061    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.482161 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.482142    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.484089 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.484064    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.484245 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.484227    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.490752 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.490723    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48df8\" (UniqueName: \"kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8\") pod \"e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") " pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.611353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.611254    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:46.754525 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.754499    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:25:46.756940 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:25:46.756913    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d87ee6b_2c8c_462a_b997_c44b9bcd4047.slice/crio-fc86e98baa46b6c3d92a42613edeec911814d3d4a1e491ca50169ab06b34ba48 WatchSource:0}: Error finding container fc86e98baa46b6c3d92a42613edeec911814d3d4a1e491ca50169ab06b34ba48: Status 404 returned error can't find the container with id fc86e98baa46b6c3d92a42613edeec911814d3d4a1e491ca50169ab06b34ba48
Jul 14 22:25:46.955679 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.955637    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" event={"ID":"4d87ee6b-2c8c-462a-b997-c44b9bcd4047","Type":"ContainerStarted","Data":"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"}
Jul 14 22:25:46.955679 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.955686    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" event={"ID":"4d87ee6b-2c8c-462a-b997-c44b9bcd4047","Type":"ContainerStarted","Data":"fc86e98baa46b6c3d92a42613edeec911814d3d4a1e491ca50169ab06b34ba48"}
Jul 14 22:25:46.972797 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:46.972736    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" podStartSLOduration=0.97271811 podStartE2EDuration="972.71811ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:25:46.9722493 +0000 UTC m=+2087.660225410" watchObservedRunningTime="2026-07-14 22:25:46.97271811 +0000 UTC m=+2087.660694203"
Jul 14 22:25:56.611564 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:56.611519    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:56.612094 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:56.611795    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:56.624617 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:56.624588    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:57.017284 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:57.017252    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:25:59.971048 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:59.971022    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:25:59.981903 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:59.981872    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:25:59.987749 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:59.987719    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:25:59.995499 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:25:59.995473    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:26:17.266520 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.266485    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:26:17.267127 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.266734    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" podUID="e12396de-a551-4e03-8077-c4c2c6e76094" containerName="maas-api" containerID="cri-o://01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a" gracePeriod=30
Jul 14 22:26:17.295481 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.295446    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:26:17.528013 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.527931    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:26:17.567054 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.567010    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdbxs\" (UniqueName: \"kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs\") pod \"e12396de-a551-4e03-8077-c4c2c6e76094\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") "
Jul 14 22:26:17.567226 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.567100    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") pod \"e12396de-a551-4e03-8077-c4c2c6e76094\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") "
Jul 14 22:26:17.567226 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.567142    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca\") pod \"e12396de-a551-4e03-8077-c4c2c6e76094\" (UID: \"e12396de-a551-4e03-8077-c4c2c6e76094\") "
Jul 14 22:26:17.567560 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.567529    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e12396de-a551-4e03-8077-c4c2c6e76094" (UID: "e12396de-a551-4e03-8077-c4c2c6e76094"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:26:17.569334 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.569296    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e12396de-a551-4e03-8077-c4c2c6e76094" (UID: "e12396de-a551-4e03-8077-c4c2c6e76094"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:26:17.569486 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.569392    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs" (OuterVolumeSpecName: "kube-api-access-sdbxs") pod "e12396de-a551-4e03-8077-c4c2c6e76094" (UID: "e12396de-a551-4e03-8077-c4c2c6e76094"). InnerVolumeSpecName "kube-api-access-sdbxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:26:17.667965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.667920    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdbxs\" (UniqueName: \"kubernetes.io/projected/e12396de-a551-4e03-8077-c4c2c6e76094-kube-api-access-sdbxs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:17.667965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.667956    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e12396de-a551-4e03-8077-c4c2c6e76094-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:17.667965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:17.667966    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e12396de-a551-4e03-8077-c4c2c6e76094-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:18.113117 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.113081    2570 generic.go:356] "Generic (PLEG): container finished" podID="e12396de-a551-4e03-8077-c4c2c6e76094" containerID="01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a" exitCode=0
Jul 14 22:26:18.113294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.113133    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"
Jul 14 22:26:18.113294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.113132    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" event={"ID":"e12396de-a551-4e03-8077-c4c2c6e76094","Type":"ContainerDied","Data":"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"}
Jul 14 22:26:18.113294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.113264    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7" event={"ID":"e12396de-a551-4e03-8077-c4c2c6e76094","Type":"ContainerDied","Data":"8b7b270f138866410083e4103f0bed9c73907cd56cb243ed3cebd0671b9ab9a9"}
Jul 14 22:26:18.113419 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.113296    2570 scope.go:117] "RemoveContainer" containerID="01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"
Jul 14 22:26:18.122434 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.122413    2570 scope.go:117] "RemoveContainer" containerID="01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"
Jul 14 22:26:18.122770 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:26:18.122751    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a\": container with ID starting with 01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a not found: ID does not exist" containerID="01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"
Jul 14 22:26:18.122893 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.122780    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a"} err="failed to get container status \"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a\": rpc error: code = NotFound desc = could not find container \"01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a\": container with ID starting with 01b7071a4b52f539cd6dad2a5a1059e87777360ca6caaf19d437871322c7f63a not found: ID does not exist"
Jul 14 22:26:18.133418 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.133321    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:26:18.136439 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:18.136408    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ba078dbe-85475857bc-d9vj7"]
Jul 14 22:26:19.881825 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:19.881782    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e12396de-a551-4e03-8077-c4c2c6e76094" path="/var/lib/kubelet/pods/e12396de-a551-4e03-8077-c4c2c6e76094/volumes"
Jul 14 22:26:21.808930 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:21.808900    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:26:26.791726 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:26.791692    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:26:26.888795 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:26.888761    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:26:41.812560 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:41.812514    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" podUID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" containerName="main" containerID="cri-o://3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb" gracePeriod=45
Jul 14 22:26:42.074486 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.074416    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:26:42.198278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198234    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198291    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198508 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198330    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198508 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198358    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198508 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198379    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198508 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198463    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48df8\" (UniqueName: \"kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8\") pod \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\" (UID: \"4d87ee6b-2c8c-462a-b997-c44b9bcd4047\") "
Jul 14 22:26:42.198690 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198631    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home" (OuterVolumeSpecName: "home") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:26:42.198751 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198649    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache" (OuterVolumeSpecName: "model-cache") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:26:42.198751 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198708    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:26:42.198897 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198824    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:42.198897 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198840    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:42.198897 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.198851    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:42.200697 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.200664    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8" (OuterVolumeSpecName: "kube-api-access-48df8") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "kube-api-access-48df8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:26:42.200797 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.200711    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm" (OuterVolumeSpecName: "dshm") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:26:42.200797 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.200739    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4d87ee6b-2c8c-462a-b997-c44b9bcd4047" (UID: "4d87ee6b-2c8c-462a-b997-c44b9bcd4047"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:26:42.239639 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.239600    2570 generic.go:356] "Generic (PLEG): container finished" podID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" containerID="3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb" exitCode=0
Jul 14 22:26:42.239831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.239656    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"
Jul 14 22:26:42.239831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.239689    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" event={"ID":"4d87ee6b-2c8c-462a-b997-c44b9bcd4047","Type":"ContainerDied","Data":"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"}
Jul 14 22:26:42.239831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.239737    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn" event={"ID":"4d87ee6b-2c8c-462a-b997-c44b9bcd4047","Type":"ContainerDied","Data":"fc86e98baa46b6c3d92a42613edeec911814d3d4a1e491ca50169ab06b34ba48"}
Jul 14 22:26:42.239831 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.239761    2570 scope.go:117] "RemoveContainer" containerID="3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"
Jul 14 22:26:42.249689 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.249670    2570 scope.go:117] "RemoveContainer" containerID="3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"
Jul 14 22:26:42.250117 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:26:42.250094    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb\": container with ID starting with 3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb not found: ID does not exist" containerID="3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"
Jul 14 22:26:42.250208 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.250127    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb"} err="failed to get container status \"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb\": rpc error: code = NotFound desc = could not find container \"3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb\": container with ID starting with 3a0cb3593cb33bfec7268fc993781adfcb2641eb006c0ab6eae4ae71fa81d7eb not found: ID does not exist"
Jul 14 22:26:42.261563 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.261527    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:26:42.265886 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.265858    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ba078dbe/e2e-lifecycle-model-ba078dbe-kserve-c9945fbbc-74pxn"]
Jul 14 22:26:42.300353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.300310    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48df8\" (UniqueName: \"kubernetes.io/projected/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-kube-api-access-48df8\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:42.300353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.300345    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:42.300353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:42.300357    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d87ee6b-2c8c-462a-b997-c44b9bcd4047-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:26:43.875973 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:26:43.875935    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" path="/var/lib/kubelet/pods/4d87ee6b-2c8c-462a-b997-c44b9bcd4047/volumes"
Jul 14 22:27:03.562958 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:03.562916    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:03.919161 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:03.919065    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:09.540148 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:09.540106    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:09.639273 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:09.639236    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:14.106635 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:14.106581    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:14.334031 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:14.333969    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:15.380730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:15.380696    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:15.486996 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:15.486961    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:15.581728 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:15.581696    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:16.367123 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:16.367081    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:16.467116 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:16.467072    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:16.572245 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:16.572208    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:18.034355 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:18.034315    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:23.786994 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:23.786942    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:23.860932 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:23.860900    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:24.468537 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:24.468502    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:30.463670 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:30.463629    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:30.532744 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:30.532708    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:30.609908 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:30.609869    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:30.705732 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:30.705699    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:41.251181 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:41.251118    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:27:41.680299 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:27:41.680174    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:02.615685 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:02.615646    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:09.259914 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:09.259874    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:09.349531 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:09.349497    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:09.666877 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:09.666781    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:09.997444 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:09.997401    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:10.106303 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:10.106269    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:12.974927 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.974890    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:28:12.975339 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975309    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" containerName="main"
Jul 14 22:28:12.975339 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975322    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" containerName="main"
Jul 14 22:28:12.975339 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975338    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12396de-a551-4e03-8077-c4c2c6e76094" containerName="maas-api"
Jul 14 22:28:12.975446 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975344    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12396de-a551-4e03-8077-c4c2c6e76094" containerName="maas-api"
Jul 14 22:28:12.975446 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975411    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d87ee6b-2c8c-462a-b997-c44b9bcd4047" containerName="main"
Jul 14 22:28:12.975446 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.975419    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e12396de-a551-4e03-8077-c4c2c6e76094" containerName="maas-api"
Jul 14 22:28:12.978327 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.978310    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:12.981293 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.981266    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-6a0cfb-serving-cert\""
Jul 14 22:28:12.998041 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:12.998013    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:28:13.109058 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.109015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.109249 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.109093    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.109249 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.109126    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4vwx\" (UniqueName: \"kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.209823 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.209778    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.209995 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.209888    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4vwx\" (UniqueName: \"kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.209995 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.209926    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.210563 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.210537    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.212355 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.212332    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.220669 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.220638    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4vwx\" (UniqueName: \"kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx\") pod \"maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.288583 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.288496    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:13.325666 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.325624    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:13.444988 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.444958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:28:13.446029 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:28:13.445997    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b0053c_93eb_4071_8f60_83f8e679620d.slice/crio-870d034c1a99a3a3dba0fe43ba8952ffc1e47d0c11add5f22313048c5a916d68 WatchSource:0}: Error finding container 870d034c1a99a3a3dba0fe43ba8952ffc1e47d0c11add5f22313048c5a916d68: Status 404 returned error can't find the container with id 870d034c1a99a3a3dba0fe43ba8952ffc1e47d0c11add5f22313048c5a916d68
Jul 14 22:28:13.452951 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.452919    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:13.677933 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.677842    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" event={"ID":"13b0053c-93eb-4071-8f60-83f8e679620d","Type":"ContainerStarted","Data":"870d034c1a99a3a3dba0fe43ba8952ffc1e47d0c11add5f22313048c5a916d68"}
Jul 14 22:28:13.728762 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.728731    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:13.836267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:13.836231    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:14.684635 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:14.684597    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" event={"ID":"13b0053c-93eb-4071-8f60-83f8e679620d","Type":"ContainerStarted","Data":"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"}
Jul 14 22:28:14.685125 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:14.684722    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:17.370788 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:17.370750    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:17.716243 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:17.716205    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:18.073264 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:18.073167    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:18.540411 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:18.540382    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:20.694470 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.694442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:28:20.713750 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.713691    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" podStartSLOduration=7.706136738 podStartE2EDuration="8.713673982s" totalImagesPullingTime="1.007537244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:28:14.722520449 +0000 UTC m=+2235.410496541" watchObservedRunningTime="2026-07-14 22:28:20.713673982 +0000 UTC m=+2241.401650074"
Jul 14 22:28:20.918450 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.918380    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:20.991773 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.991688    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:28:20.995373 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.995348    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:20.997747 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:20.997719    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b4e43e-serving-cert\""
Jul 14 22:28:21.008360 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.008327    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:28:21.022669 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.022632    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:21.093015 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.092964    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.093204 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.093026    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp9g5\" (UniqueName: \"kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.093204 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.093053    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.105485 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.105459    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:21.194062 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.194025    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.194243 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.194068    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp9g5\" (UniqueName: \"kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.194243 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.194092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.194743 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.194720    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.196505 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.196477    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.213742 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.213708    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:21.216606 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.216573    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp9g5\" (UniqueName: \"kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5\") pod \"maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.309589 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.309494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:21.467890 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.467860    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:28:21.469625 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:28:21.469595    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e80035_32b2_4a8e_a882_be398856b7e4.slice/crio-d17b81647e541cd52cfe0348755dc8aa033557c77034f1d8b52813885bddd103 WatchSource:0}: Error finding container d17b81647e541cd52cfe0348755dc8aa033557c77034f1d8b52813885bddd103: Status 404 returned error can't find the container with id d17b81647e541cd52cfe0348755dc8aa033557c77034f1d8b52813885bddd103
Jul 14 22:28:21.637247 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.637168    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:21.720757 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.720724    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" event={"ID":"68e80035-32b2-4a8e-a882-be398856b7e4","Type":"ContainerStarted","Data":"d17b81647e541cd52cfe0348755dc8aa033557c77034f1d8b52813885bddd103"}
Jul 14 22:28:21.737767 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:21.737695    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:22.727213 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:22.727179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" event={"ID":"68e80035-32b2-4a8e-a882-be398856b7e4","Type":"ContainerStarted","Data":"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"}
Jul 14 22:28:22.727652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:22.727296    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:22.763294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:22.763241    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" podStartSLOduration=2.349540821 podStartE2EDuration="2.763222591s" totalImagesPullingTime="413.68177ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:28:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:28:22.760703021 +0000 UTC m=+2243.448679141" watchObservedRunningTime="2026-07-14 22:28:22.763222591 +0000 UTC m=+2243.451198686"
Jul 14 22:28:25.424363 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:25.424334    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:28.739261 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:28.739226    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:28:29.980933 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:29.980900    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:34.732299 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:34.732250    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:34.845022 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:34.844987    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:36.724149 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:36.724112    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:42.601948 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:42.601903    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:28:44.086958 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:28:44.086920    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:09.090777 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.090740    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:09.094377 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.094356    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.097855 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.097829    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b4e43e\"/\"kube-root-ca.crt\""
Jul 14 22:29:09.097855 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.097841    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b4e43e\"/\"default-dockercfg-8j2dg\""
Jul 14 22:29:09.098065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.097866    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b4e43e\"/\"test-model-b4e43e-kserve-self-signed-certs\""
Jul 14 22:29:09.098065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.097887    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b4e43e\"/\"openshift-service-ca.crt\""
Jul 14 22:29:09.105475 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.105449    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:09.121594 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.121556    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:09.152851 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.152783    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.153060 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.152957    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqt2g\" (UniqueName: \"kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.153060 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.153013    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.153188 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.153082    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.153188 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.153137    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.153188 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.153166    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254297 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254297 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254302    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqt2g\" (UniqueName: \"kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254387    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254423    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254579 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254448    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.254891 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254855    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.255030 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254912    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.255030 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.254915    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.256726 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.256700    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.256997 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.256980    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.264423 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.264393    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqt2g\" (UniqueName: \"kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g\") pod \"test-model-b4e43e-kserve-9f74d4f8c-mwq25\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") " pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.404572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.404476    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:09.561727 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.561691    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:09.564187 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:29:09.564149    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbe6cf1d_408c_4f20_bb53_0eac3640ddae.slice/crio-4d0c377e7ca933cebcf82e51951b095d7d43aa0f5dcc26839b5a86985eab6881 WatchSource:0}: Error finding container 4d0c377e7ca933cebcf82e51951b095d7d43aa0f5dcc26839b5a86985eab6881: Status 404 returned error can't find the container with id 4d0c377e7ca933cebcf82e51951b095d7d43aa0f5dcc26839b5a86985eab6881
Jul 14 22:29:09.961353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.961318    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" event={"ID":"bbe6cf1d-408c-4f20-bb53-0eac3640ddae","Type":"ContainerStarted","Data":"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"}
Jul 14 22:29:09.961353 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.961359    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" event={"ID":"bbe6cf1d-408c-4f20-bb53-0eac3640ddae","Type":"ContainerStarted","Data":"4d0c377e7ca933cebcf82e51951b095d7d43aa0f5dcc26839b5a86985eab6881"}
Jul 14 22:29:09.995828 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:09.995742    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" podStartSLOduration=0.995722759 podStartE2EDuration="995.722759ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:29:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:29:09.994436545 +0000 UTC m=+2290.682412660" watchObservedRunningTime="2026-07-14 22:29:09.995722759 +0000 UTC m=+2290.683698851"
Jul 14 22:29:17.617472 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:17.617431    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:19.405591 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:19.405553    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:19.405591 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:19.405599    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:19.418142 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:19.418106    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:20.019669 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:20.019640    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:34.568858 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:34.568766    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:35.622431 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:35.622395    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:35.680273 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:35.680236    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:36.318139 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:36.318100    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:36.421253 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:36.421222    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:38.999391 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:38.999354    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:39.144943 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:39.144908    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:46.442362 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.442324    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:46.539758 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.539720    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:46.730503 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.730400    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:29:46.731006 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.730967    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" podUID="13b0053c-93eb-4071-8f60-83f8e679620d" containerName="maas-api" containerID="cri-o://29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf" gracePeriod=30
Jul 14 22:29:46.776005 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.775969    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:46.999499 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:46.999467    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:29:47.110774 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.110733    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls\") pod \"13b0053c-93eb-4071-8f60-83f8e679620d\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") "
Jul 14 22:29:47.110774 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.110781    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4vwx\" (UniqueName: \"kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx\") pod \"13b0053c-93eb-4071-8f60-83f8e679620d\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") "
Jul 14 22:29:47.111070 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.110907    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca\") pod \"13b0053c-93eb-4071-8f60-83f8e679620d\" (UID: \"13b0053c-93eb-4071-8f60-83f8e679620d\") "
Jul 14 22:29:47.111298 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.111262    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13b0053c-93eb-4071-8f60-83f8e679620d" (UID: "13b0053c-93eb-4071-8f60-83f8e679620d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:29:47.113016 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.112984    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13b0053c-93eb-4071-8f60-83f8e679620d" (UID: "13b0053c-93eb-4071-8f60-83f8e679620d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:29:47.113127 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.112993    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx" (OuterVolumeSpecName: "kube-api-access-t4vwx") pod "13b0053c-93eb-4071-8f60-83f8e679620d" (UID: "13b0053c-93eb-4071-8f60-83f8e679620d"). InnerVolumeSpecName "kube-api-access-t4vwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:29:47.150372 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.150336    2570 generic.go:356] "Generic (PLEG): container finished" podID="13b0053c-93eb-4071-8f60-83f8e679620d" containerID="29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf" exitCode=0
Jul 14 22:29:47.150572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.150386    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"
Jul 14 22:29:47.150572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.150406    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" event={"ID":"13b0053c-93eb-4071-8f60-83f8e679620d","Type":"ContainerDied","Data":"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"}
Jul 14 22:29:47.150572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.150458    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j" event={"ID":"13b0053c-93eb-4071-8f60-83f8e679620d","Type":"ContainerDied","Data":"870d034c1a99a3a3dba0fe43ba8952ffc1e47d0c11add5f22313048c5a916d68"}
Jul 14 22:29:47.150572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.150483    2570 scope.go:117] "RemoveContainer" containerID="29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"
Jul 14 22:29:47.160141 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.160119    2570 scope.go:117] "RemoveContainer" containerID="29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"
Jul 14 22:29:47.160486 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:29:47.160465    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf\": container with ID starting with 29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf not found: ID does not exist" containerID="29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"
Jul 14 22:29:47.160547 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.160500    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf"} err="failed to get container status \"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf\": rpc error: code = NotFound desc = could not find container \"29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf\": container with ID starting with 29a96a8836edf5c07f269073abd1d10a3057969848a20c748a8b5a9b403c6bdf not found: ID does not exist"
Jul 14 22:29:47.177197 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.177158    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:29:47.185847 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.185814    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-6a0cfb-577d757686-cr82j"]
Jul 14 22:29:47.212228 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.212194    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13b0053c-93eb-4071-8f60-83f8e679620d-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:47.212228 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.212228    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13b0053c-93eb-4071-8f60-83f8e679620d-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:47.212434 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.212241    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4vwx\" (UniqueName: \"kubernetes.io/projected/13b0053c-93eb-4071-8f60-83f8e679620d-kube-api-access-t4vwx\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:47.876266 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:47.876233    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13b0053c-93eb-4071-8f60-83f8e679620d" path="/var/lib/kubelet/pods/13b0053c-93eb-4071-8f60-83f8e679620d/volumes"
Jul 14 22:29:50.644075 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.644021    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" podUID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" containerName="main" containerID="cri-o://4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279" gracePeriod=45
Jul 14 22:29:50.855014 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.854978    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:50.905522 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.905445    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:50.948669 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948629    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.948669 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948673    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.948974 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948726    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqt2g\" (UniqueName: \"kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.948974 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948777    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.948974 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948844    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.948974 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948884    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs\") pod \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\" (UID: \"bbe6cf1d-408c-4f20-bb53-0eac3640ddae\") "
Jul 14 22:29:50.949204 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.948992    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:29:50.949204 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.949134    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home" (OuterVolumeSpecName: "home") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:29:50.949319 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.949257    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:50.949319 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.949276    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:50.949428 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.949369    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache" (OuterVolumeSpecName: "model-cache") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:29:50.951109 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.951074    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm" (OuterVolumeSpecName: "dshm") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:29:50.951109 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.951088    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g" (OuterVolumeSpecName: "kube-api-access-gqt2g") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "kube-api-access-gqt2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:29:50.951267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:50.951144    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bbe6cf1d-408c-4f20-bb53-0eac3640ddae" (UID: "bbe6cf1d-408c-4f20-bb53-0eac3640ddae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:29:51.049730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.049688    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:51.049730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.049722    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:51.049730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.049730    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:51.049730 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.049738    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqt2g\" (UniqueName: \"kubernetes.io/projected/bbe6cf1d-408c-4f20-bb53-0eac3640ddae-kube-api-access-gqt2g\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:29:51.173059 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.172963    2570 generic.go:356] "Generic (PLEG): container finished" podID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" containerID="4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279" exitCode=0
Jul 14 22:29:51.173059 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.173009    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"
Jul 14 22:29:51.173299 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.173057    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" event={"ID":"bbe6cf1d-408c-4f20-bb53-0eac3640ddae","Type":"ContainerDied","Data":"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"}
Jul 14 22:29:51.173299 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.173105    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25" event={"ID":"bbe6cf1d-408c-4f20-bb53-0eac3640ddae","Type":"ContainerDied","Data":"4d0c377e7ca933cebcf82e51951b095d7d43aa0f5dcc26839b5a86985eab6881"}
Jul 14 22:29:51.173299 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.173127    2570 scope.go:117] "RemoveContainer" containerID="4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"
Jul 14 22:29:51.182960 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.182941    2570 scope.go:117] "RemoveContainer" containerID="4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"
Jul 14 22:29:51.183292 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:29:51.183271    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279\": container with ID starting with 4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279 not found: ID does not exist" containerID="4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"
Jul 14 22:29:51.183364 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.183300    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279"} err="failed to get container status \"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279\": rpc error: code = NotFound desc = could not find container \"4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279\": container with ID starting with 4792a91d5fd5daa055b8765771724f62bd9a240b78ce33d619c13f9c26554279 not found: ID does not exist"
Jul 14 22:29:51.195768 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.195730    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:51.202132 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.202097    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b4e43e/test-model-b4e43e-kserve-9f74d4f8c-mwq25"]
Jul 14 22:29:51.875576 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:51.875539    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" path="/var/lib/kubelet/pods/bbe6cf1d-408c-4f20-bb53-0eac3640ddae/volumes"
Jul 14 22:29:56.986347 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:56.986316    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:29:57.120769 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:29:57.120730    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:00.144979 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.144942    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"]
Jul 14 22:30:00.145374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145315    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" containerName="main"
Jul 14 22:30:00.145374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145326    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" containerName="main"
Jul 14 22:30:00.145374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145352    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13b0053c-93eb-4071-8f60-83f8e679620d" containerName="maas-api"
Jul 14 22:30:00.145374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145357    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="13b0053c-93eb-4071-8f60-83f8e679620d" containerName="maas-api"
Jul 14 22:30:00.145507 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145431    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe6cf1d-408c-4f20-bb53-0eac3640ddae" containerName="main"
Jul 14 22:30:00.145507 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.145443    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="13b0053c-93eb-4071-8f60-83f8e679620d" containerName="maas-api"
Jul 14 22:30:00.150117 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.150092    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"
Jul 14 22:30:00.153061 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.153037    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4jlcq\""
Jul 14 22:30:00.161457 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.161421    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"]
Jul 14 22:30:00.164216 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.164186    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"
Jul 14 22:30:00.316139 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.316045    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"]
Jul 14 22:30:00.319851 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:30:00.319822    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6f8298_9032_4a2a_b458_fbe5a2316b04.slice/crio-894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7 WatchSource:0}: Error finding container 894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7: Status 404 returned error can't find the container with id 894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7
Jul 14 22:30:00.322187 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:00.322169    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:30:01.225965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:01.225925    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk" event={"ID":"1c6f8298-9032-4a2a-b458-fbe5a2316b04","Type":"ContainerStarted","Data":"894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7"}
Jul 14 22:30:01.597439 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:01.597341    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:01.863215 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:01.863116    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:01.993351 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:01.993306    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"]
Jul 14 22:30:02.003009 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.002973    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:02.003686 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.003646    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"]
Jul 14 22:30:02.054154 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.054097    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx\" (UID: \"706c9a81-9bb9-4d05-a269-90af42f6ed65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:02.155045 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.154953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx\" (UID: \"706c9a81-9bb9-4d05-a269-90af42f6ed65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:02.155734 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.155705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx\" (UID: \"706c9a81-9bb9-4d05-a269-90af42f6ed65\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:02.317886 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.317843    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:02.550869 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:30:02.550833    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706c9a81_9bb9_4d05_a269_90af42f6ed65.slice/crio-1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d WatchSource:0}: Error finding container 1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d: Status 404 returned error can't find the container with id 1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d
Jul 14 22:30:02.555838 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.555774    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"]
Jul 14 22:30:02.681369 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.681327    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:02.866638 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:02.866586    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:03.238444 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:03.238408    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk" event={"ID":"1c6f8298-9032-4a2a-b458-fbe5a2316b04","Type":"ContainerStarted","Data":"9048f56a7118f26fed90724266af3eb8bf620afe9b5199bafb55fc49face43fe"}
Jul 14 22:30:03.239984 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:03.239941    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx" event={"ID":"706c9a81-9bb9-4d05-a269-90af42f6ed65","Type":"ContainerStarted","Data":"3401fbc88db38693adef35b4596ae04ce9d97f91fedce05e95d9993cdaf6cfd0"}
Jul 14 22:30:03.240135 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:03.240004    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx" event={"ID":"706c9a81-9bb9-4d05-a269-90af42f6ed65","Type":"ContainerStarted","Data":"1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d"}
Jul 14 22:30:03.258571 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:03.258508    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk" podStartSLOduration=1.115375219 podStartE2EDuration="3.258487706s" totalImagesPullingTime="2.143112487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:30:03.256539311 +0000 UTC m=+2343.944515422" watchObservedRunningTime="2026-07-14 22:30:03.258487706 +0000 UTC m=+2343.946463845"
Jul 14 22:30:03.274348 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:03.274283    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx" podStartSLOduration=2.274263644 podStartE2EDuration="2.274263644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:30:03.272047546 +0000 UTC m=+2343.960023649" watchObservedRunningTime="2026-07-14 22:30:03.274263644 +0000 UTC m=+2343.962239736"
Jul 14 22:30:04.247071 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:04.247037    2570 generic.go:356] "Generic (PLEG): container finished" podID="1c6f8298-9032-4a2a-b458-fbe5a2316b04" containerID="9048f56a7118f26fed90724266af3eb8bf620afe9b5199bafb55fc49face43fe" exitCode=0
Jul 14 22:30:04.247529 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:04.247129    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk" event={"ID":"1c6f8298-9032-4a2a-b458-fbe5a2316b04","Type":"ContainerDied","Data":"9048f56a7118f26fed90724266af3eb8bf620afe9b5199bafb55fc49face43fe"}
Jul 14 22:30:04.248694 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:04.248671    2570 generic.go:356] "Generic (PLEG): container finished" podID="706c9a81-9bb9-4d05-a269-90af42f6ed65" containerID="3401fbc88db38693adef35b4596ae04ce9d97f91fedce05e95d9993cdaf6cfd0" exitCode=0
Jul 14 22:30:04.248781 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:04.248709    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx" event={"ID":"706c9a81-9bb9-4d05-a269-90af42f6ed65","Type":"ContainerDied","Data":"3401fbc88db38693adef35b4596ae04ce9d97f91fedce05e95d9993cdaf6cfd0"}
Jul 14 22:30:05.402529 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:05.402497    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"
Jul 14 22:30:05.436047 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:05.436011    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:05.485566 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:05.485533    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca\") pod \"706c9a81-9bb9-4d05-a269-90af42f6ed65\" (UID: \"706c9a81-9bb9-4d05-a269-90af42f6ed65\") "
Jul 14 22:30:05.485970 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:05.485945    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "706c9a81-9bb9-4d05-a269-90af42f6ed65" (UID: "706c9a81-9bb9-4d05-a269-90af42f6ed65"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:30:05.586884 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:05.586755    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/706c9a81-9bb9-4d05-a269-90af42f6ed65-maas-api-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:30:06.262923 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.262880    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx"
Jul 14 22:30:06.262923 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.262916    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx" event={"ID":"706c9a81-9bb9-4d05-a269-90af42f6ed65","Type":"ContainerDied","Data":"1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d"}
Jul 14 22:30:06.263147 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.262968    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b0dd956d12bd84c6ad44babf2cbc085d0ed7fde889d2e5be887fe27fff7654d"
Jul 14 22:30:06.264652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.264623    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk" event={"ID":"1c6f8298-9032-4a2a-b458-fbe5a2316b04","Type":"ContainerDied","Data":"894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7"}
Jul 14 22:30:06.264652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.264640    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29734470-npnhk"
Jul 14 22:30:06.264895 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:06.264667    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="894ed3902887c046fb145e40b02f5af291c56fefef107df9aae9ae5a1dc629d7"
Jul 14 22:30:10.072121 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:10.072087    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:12.468343 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.468302    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:30:12.469230 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.469190    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" podUID="68e80035-32b2-4a8e-a882-be398856b7e4" containerName="maas-api" containerID="cri-o://b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24" gracePeriod=30
Jul 14 22:30:12.511002 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.510970    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:12.730759 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.730666    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:30:12.855413 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.855378    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lp9g5\" (UniqueName: \"kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5\") pod \"68e80035-32b2-4a8e-a882-be398856b7e4\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") "
Jul 14 22:30:12.855592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.855480    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls\") pod \"68e80035-32b2-4a8e-a882-be398856b7e4\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") "
Jul 14 22:30:12.855592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.855502    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca\") pod \"68e80035-32b2-4a8e-a882-be398856b7e4\" (UID: \"68e80035-32b2-4a8e-a882-be398856b7e4\") "
Jul 14 22:30:12.855946 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.855921    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "68e80035-32b2-4a8e-a882-be398856b7e4" (UID: "68e80035-32b2-4a8e-a882-be398856b7e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:30:12.857623 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.857602    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "68e80035-32b2-4a8e-a882-be398856b7e4" (UID: "68e80035-32b2-4a8e-a882-be398856b7e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:30:12.857795 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.857762    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5" (OuterVolumeSpecName: "kube-api-access-lp9g5") pod "68e80035-32b2-4a8e-a882-be398856b7e4" (UID: "68e80035-32b2-4a8e-a882-be398856b7e4"). InnerVolumeSpecName "kube-api-access-lp9g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:30:12.956972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.956924    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68e80035-32b2-4a8e-a882-be398856b7e4-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:30:12.956972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.956963    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68e80035-32b2-4a8e-a882-be398856b7e4-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:30:12.956972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:12.956976    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp9g5\" (UniqueName: \"kubernetes.io/projected/68e80035-32b2-4a8e-a882-be398856b7e4-kube-api-access-lp9g5\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:30:13.303050 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.303014    2570 generic.go:356] "Generic (PLEG): container finished" podID="68e80035-32b2-4a8e-a882-be398856b7e4" containerID="b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24" exitCode=0
Jul 14 22:30:13.303278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.303063    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"
Jul 14 22:30:13.303278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.303101    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" event={"ID":"68e80035-32b2-4a8e-a882-be398856b7e4","Type":"ContainerDied","Data":"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"}
Jul 14 22:30:13.303278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.303160    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj" event={"ID":"68e80035-32b2-4a8e-a882-be398856b7e4","Type":"ContainerDied","Data":"d17b81647e541cd52cfe0348755dc8aa033557c77034f1d8b52813885bddd103"}
Jul 14 22:30:13.303278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.303190    2570 scope.go:117] "RemoveContainer" containerID="b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"
Jul 14 22:30:13.312780 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.312762    2570 scope.go:117] "RemoveContainer" containerID="b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"
Jul 14 22:30:13.313193 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:30:13.313164    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24\": container with ID starting with b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24 not found: ID does not exist" containerID="b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"
Jul 14 22:30:13.313298 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.313199    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24"} err="failed to get container status \"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24\": rpc error: code = NotFound desc = could not find container \"b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24\": container with ID starting with b2fee36b5ae86dcd18de741d1caa45730805930ee72ca6db01ceedd42b248d24 not found: ID does not exist"
Jul 14 22:30:13.327367 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.327329    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:30:13.330737 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.330685    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b4e43e-554985bd94-kmqgj"]
Jul 14 22:30:13.875605 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:13.875560    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68e80035-32b2-4a8e-a882-be398856b7e4" path="/var/lib/kubelet/pods/68e80035-32b2-4a8e-a882-be398856b7e4/volumes"
Jul 14 22:30:17.126127 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:17.126090    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:23.123345 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:23.123303    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:23.225591 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:23.225558    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:27.891178 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:27.891124    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:28.142499 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:28.142418    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:28.562623 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:28.562588    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:29.204079 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:29.204048    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:29.384152 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:29.384121    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:29.452676 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:29.452637    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:29.773352 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:29.773316    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:32.731939 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.731899    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:32.834746 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.834713    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:32.859441 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859394    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:30:32.859903 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859884    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c6f8298-9032-4a2a-b458-fbe5a2316b04" containerName="cleanup"
Jul 14 22:30:32.860010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859905    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c6f8298-9032-4a2a-b458-fbe5a2316b04" containerName="cleanup"
Jul 14 22:30:32.860010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859924    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706c9a81-9bb9-4d05-a269-90af42f6ed65" containerName="revoke-keys"
Jul 14 22:30:32.860010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859931    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="706c9a81-9bb9-4d05-a269-90af42f6ed65" containerName="revoke-keys"
Jul 14 22:30:32.860010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859945    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68e80035-32b2-4a8e-a882-be398856b7e4" containerName="maas-api"
Jul 14 22:30:32.860010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.859953    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="68e80035-32b2-4a8e-a882-be398856b7e4" containerName="maas-api"
Jul 14 22:30:32.860267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.860044    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c6f8298-9032-4a2a-b458-fbe5a2316b04" containerName="cleanup"
Jul 14 22:30:32.860267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.860058    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="68e80035-32b2-4a8e-a882-be398856b7e4" containerName="maas-api"
Jul 14 22:30:32.860267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.860068    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="706c9a81-9bb9-4d05-a269-90af42f6ed65" containerName="revoke-keys"
Jul 14 22:30:32.864597 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.864563    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:32.867248 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.867223    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-de6a00-serving-cert\""
Jul 14 22:30:32.877908 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.877877    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:30:32.971385 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:32.971351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:33.040079 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.039991    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.040079 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.040045    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8t5r\" (UniqueName: \"kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.040312 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.040161    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.141025 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.140974    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.141213 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.141054    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.141213 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.141092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8t5r\" (UniqueName: \"kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.141213 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:30:33.141152    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-serving-cert: secret "maas-api-e2e-api-a-de6a00-serving-cert" not found
Jul 14 22:30:33.141326 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:30:33.141230    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls podName:45f075e5-c3c8-4bcd-be07-001874c2fa2c nodeName:}" failed. No retries permitted until 2026-07-14 22:30:33.641208245 +0000 UTC m=+2374.329184329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls") pod "maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" (UID: "45f075e5-c3c8-4bcd-be07-001874c2fa2c") : secret "maas-api-e2e-api-a-de6a00-serving-cert" not found
Jul 14 22:30:33.141847 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.141791    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.157873 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.157838    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8t5r\" (UniqueName: \"kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.366263 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.366175    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:33.646928 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.646796    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.649435 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.649397    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") pod \"maas-api-e2e-api-a-de6a00-844c4bb557-rnf55\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.670120 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.670081    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:33.764893 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.764857    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:33.776859 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.776787    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:33.939481 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:33.939417    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:30:33.941861 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:30:33.941830    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45f075e5_c3c8_4bcd_be07_001874c2fa2c.slice/crio-e30ca3986b87784e371974ae51d2bce99e37f53aac41f721866f5699639fc7d5 WatchSource:0}: Error finding container e30ca3986b87784e371974ae51d2bce99e37f53aac41f721866f5699639fc7d5: Status 404 returned error can't find the container with id e30ca3986b87784e371974ae51d2bce99e37f53aac41f721866f5699639fc7d5
Jul 14 22:30:34.414990 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:34.414897    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" event={"ID":"45f075e5-c3c8-4bcd-be07-001874c2fa2c","Type":"ContainerStarted","Data":"e30ca3986b87784e371974ae51d2bce99e37f53aac41f721866f5699639fc7d5"}
Jul 14 22:30:35.422661 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:35.422624    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" event={"ID":"45f075e5-c3c8-4bcd-be07-001874c2fa2c","Type":"ContainerStarted","Data":"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"}
Jul 14 22:30:35.423102 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:35.422696    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:35.442606 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:35.442553    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" podStartSLOduration=2.882584868 podStartE2EDuration="3.442534477s" totalImagesPullingTime="559.949609ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:30:35.440912555 +0000 UTC m=+2376.128888648" watchObservedRunningTime="2026-07-14 22:30:35.442534477 +0000 UTC m=+2376.130510569"
Jul 14 22:30:37.274261 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:37.274225    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:37.859140 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:37.859108    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:38.253102 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:38.253066    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:38.516882 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:38.516783    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:41.435335 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.435304    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:30:41.713014 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.712973    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:41.831384 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.831343    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:30:41.835370 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.835334    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:41.837964 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.837934    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-dc25fd-serving-cert\""
Jul 14 22:30:41.842500 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.842463    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:41.851003 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.850969    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:30:41.918418 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.918375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb7wr\" (UniqueName: \"kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:41.918619 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.918433    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:41.918619 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.918540    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:41.960966 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:41.960934    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:42.020024 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.019939    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb7wr\" (UniqueName: \"kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.020024 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.019999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.020227 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.020049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.020819 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.020776    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.022592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.022563    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.040105 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.040031    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb7wr\" (UniqueName: \"kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr\") pod \"maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.087326 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.087289    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:42.149057 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.149014    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:42.306272 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.306243    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:30:42.308411 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:30:42.308384    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c7d9ce_825a_4b93_a7a8_67990653b5dc.slice/crio-ff13b1e0c2bc8a77a022fb35c08ad9bb979159341bc960b48b699badd3b441ac WatchSource:0}: Error finding container ff13b1e0c2bc8a77a022fb35c08ad9bb979159341bc960b48b699badd3b441ac: Status 404 returned error can't find the container with id ff13b1e0c2bc8a77a022fb35c08ad9bb979159341bc960b48b699badd3b441ac
Jul 14 22:30:42.458319 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.458287    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" event={"ID":"86c7d9ce-825a-4b93-a7a8-67990653b5dc","Type":"ContainerStarted","Data":"ff13b1e0c2bc8a77a022fb35c08ad9bb979159341bc960b48b699badd3b441ac"}
Jul 14 22:30:42.606887 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.606688    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:42.710698 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:42.710662    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:43.379451 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:43.379420    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:43.465023 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:43.464989    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" event={"ID":"86c7d9ce-825a-4b93-a7a8-67990653b5dc","Type":"ContainerStarted","Data":"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"}
Jul 14 22:30:43.465445 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:43.465119    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:46.307582 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:46.307546    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:49.475626 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:49.475597    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:30:49.511995 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:49.511932    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" podStartSLOduration=8.196904663 podStartE2EDuration="8.511913568s" totalImagesPullingTime="315.008905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:30:43.494793015 +0000 UTC m=+2384.182769108" watchObservedRunningTime="2026-07-14 22:30:49.511913568 +0000 UTC m=+2390.199889662"
Jul 14 22:30:54.474927 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:54.474895    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:30:54.576651 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:30:54.576615    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:00.010113 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:00.010081    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:31:00.017877 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:00.017850    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:31:00.020941 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:00.020914    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:31:00.028343 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:00.028315    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:31:04.885994 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:04.885953    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:04.988140 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:04.988108    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:05.158948 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.158854    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:31:05.159119 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.159050    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" podUID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" containerName="maas-api" containerID="cri-o://7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a" gracePeriod=30
Jul 14 22:31:05.183893 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.183865    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:05.417269 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.417200    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:31:05.541205 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.541170    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8t5r\" (UniqueName: \"kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r\") pod \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") "
Jul 14 22:31:05.541406 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.541231    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") pod \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") "
Jul 14 22:31:05.541406 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.541290    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca\") pod \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\" (UID: \"45f075e5-c3c8-4bcd-be07-001874c2fa2c\") "
Jul 14 22:31:05.541745 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.541717    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45f075e5-c3c8-4bcd-be07-001874c2fa2c" (UID: "45f075e5-c3c8-4bcd-be07-001874c2fa2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:31:05.543335 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.543309    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45f075e5-c3c8-4bcd-be07-001874c2fa2c" (UID: "45f075e5-c3c8-4bcd-be07-001874c2fa2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:31:05.543446 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.543359    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r" (OuterVolumeSpecName: "kube-api-access-l8t5r") pod "45f075e5-c3c8-4bcd-be07-001874c2fa2c" (UID: "45f075e5-c3c8-4bcd-be07-001874c2fa2c"). InnerVolumeSpecName "kube-api-access-l8t5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:31:05.577245 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.577211    2570 generic.go:356] "Generic (PLEG): container finished" podID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" containerID="7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a" exitCode=0
Jul 14 22:31:05.577245 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.577253    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"
Jul 14 22:31:05.577450 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.577285    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" event={"ID":"45f075e5-c3c8-4bcd-be07-001874c2fa2c","Type":"ContainerDied","Data":"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"}
Jul 14 22:31:05.577450 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.577331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55" event={"ID":"45f075e5-c3c8-4bcd-be07-001874c2fa2c","Type":"ContainerDied","Data":"e30ca3986b87784e371974ae51d2bce99e37f53aac41f721866f5699639fc7d5"}
Jul 14 22:31:05.577450 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.577355    2570 scope.go:117] "RemoveContainer" containerID="7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"
Jul 14 22:31:05.594124 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.594094    2570 scope.go:117] "RemoveContainer" containerID="7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"
Jul 14 22:31:05.594541 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:05.594517    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a\": container with ID starting with 7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a not found: ID does not exist" containerID="7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"
Jul 14 22:31:05.594622 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.594555    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a"} err="failed to get container status \"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a\": rpc error: code = NotFound desc = could not find container \"7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a\": container with ID starting with 7ec21571cd3fd157bcff71b65e792f651bbbd90e106c6d729279ed6fec0c474a not found: ID does not exist"
Jul 14 22:31:05.608411 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.608363    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:31:05.612438 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.612409    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-de6a00-844c4bb557-rnf55"]
Jul 14 22:31:05.642640 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.642600    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8t5r\" (UniqueName: \"kubernetes.io/projected/45f075e5-c3c8-4bcd-be07-001874c2fa2c-kube-api-access-l8t5r\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:05.642640 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.642637    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45f075e5-c3c8-4bcd-be07-001874c2fa2c-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:05.642640 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.642648    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45f075e5-c3c8-4bcd-be07-001874c2fa2c-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:05.875623 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:05.875590    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" path="/var/lib/kubelet/pods/45f075e5-c3c8-4bcd-be07-001874c2fa2c/volumes"
Jul 14 22:31:09.511872 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:09.511834    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:15.885758 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:15.885709    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:15.990631 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:15.990595    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:20.247016 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.246979    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:20.481861 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.481793    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:20.603667 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.603582    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"]
Jul 14 22:31:20.604006 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.603992    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" containerName="maas-api"
Jul 14 22:31:20.604052 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.604007    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" containerName="maas-api"
Jul 14 22:31:20.604091 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.604083    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="45f075e5-c3c8-4bcd-be07-001874c2fa2c" containerName="maas-api"
Jul 14 22:31:20.607280 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.607262    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:20.610376 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.610350    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4jlcq\""
Jul 14 22:31:20.616614 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.616581    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"]
Jul 14 22:31:20.776055 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.776020    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd\" (UID: \"84193bca-3df3-400c-a259-0e5f0b9beef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:20.876754 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.876630    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd\" (UID: \"84193bca-3df3-400c-a259-0e5f0b9beef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:20.877350 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.877330    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd\" (UID: \"84193bca-3df3-400c-a259-0e5f0b9beef7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:20.918747 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:20.918706    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:21.094924 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:21.094835    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"]
Jul 14 22:31:21.098114 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:31:21.098075    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84193bca_3df3_400c_a259_0e5f0b9beef7.slice/crio-dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5 WatchSource:0}: Error finding container dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5: Status 404 returned error can't find the container with id dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5
Jul 14 22:31:21.663622 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:21.663587    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd" event={"ID":"84193bca-3df3-400c-a259-0e5f0b9beef7","Type":"ContainerStarted","Data":"04a5c7b71f144eaad08644b736c7891384a91a3aca5d40e1b71a1c645a33f041"}
Jul 14 22:31:21.664085 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:21.663644    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd" event={"ID":"84193bca-3df3-400c-a259-0e5f0b9beef7","Type":"ContainerStarted","Data":"dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5"}
Jul 14 22:31:22.669915 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:22.669883    2570 generic.go:356] "Generic (PLEG): container finished" podID="84193bca-3df3-400c-a259-0e5f0b9beef7" containerID="04a5c7b71f144eaad08644b736c7891384a91a3aca5d40e1b71a1c645a33f041" exitCode=0
Jul 14 22:31:22.670314 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:22.669928    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd" event={"ID":"84193bca-3df3-400c-a259-0e5f0b9beef7","Type":"ContainerDied","Data":"04a5c7b71f144eaad08644b736c7891384a91a3aca5d40e1b71a1c645a33f041"}
Jul 14 22:31:23.817227 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:23.817200    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:23.899587 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:23.899551    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca\") pod \"84193bca-3df3-400c-a259-0e5f0b9beef7\" (UID: \"84193bca-3df3-400c-a259-0e5f0b9beef7\") "
Jul 14 22:31:23.899963 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:23.899938    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84193bca-3df3-400c-a259-0e5f0b9beef7" (UID: "84193bca-3df3-400c-a259-0e5f0b9beef7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:31:24.000208 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:24.000172    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84193bca-3df3-400c-a259-0e5f0b9beef7-maas-api-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:24.680862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:24.680833    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd"
Jul 14 22:31:24.680862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:24.680836    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd" event={"ID":"84193bca-3df3-400c-a259-0e5f0b9beef7","Type":"ContainerDied","Data":"dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5"}
Jul 14 22:31:24.681077 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:24.680884    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd4f0cde27fb852975687e90dcb463223c15b0745ae8cd3891dcf67b961c52b5"
Jul 14 22:31:31.066707 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.066669    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:31:31.067278 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.066945    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" podUID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" containerName="maas-api" containerID="cri-o://31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f" gracePeriod=30
Jul 14 22:31:31.102098 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.102065    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:31.327943 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.327861    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:31:31.360651 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.360617    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca\") pod \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") "
Jul 14 22:31:31.360897 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.360711    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls\") pod \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") "
Jul 14 22:31:31.360897 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.360752    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb7wr\" (UniqueName: \"kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr\") pod \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\" (UID: \"86c7d9ce-825a-4b93-a7a8-67990653b5dc\") "
Jul 14 22:31:31.361115 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.361092    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86c7d9ce-825a-4b93-a7a8-67990653b5dc" (UID: "86c7d9ce-825a-4b93-a7a8-67990653b5dc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:31:31.362962 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.362935    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr" (OuterVolumeSpecName: "kube-api-access-mb7wr") pod "86c7d9ce-825a-4b93-a7a8-67990653b5dc" (UID: "86c7d9ce-825a-4b93-a7a8-67990653b5dc"). InnerVolumeSpecName "kube-api-access-mb7wr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:31:31.362962 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.362947    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86c7d9ce-825a-4b93-a7a8-67990653b5dc" (UID: "86c7d9ce-825a-4b93-a7a8-67990653b5dc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:31:31.462006 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.461966    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86c7d9ce-825a-4b93-a7a8-67990653b5dc-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:31.462006 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.461999    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86c7d9ce-825a-4b93-a7a8-67990653b5dc-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:31.462006 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.462012    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb7wr\" (UniqueName: \"kubernetes.io/projected/86c7d9ce-825a-4b93-a7a8-67990653b5dc-kube-api-access-mb7wr\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:31:31.716480 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.716444    2570 generic.go:356] "Generic (PLEG): container finished" podID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" containerID="31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f" exitCode=0
Jul 14 22:31:31.716706 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.716503    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"
Jul 14 22:31:31.716706 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.716537    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" event={"ID":"86c7d9ce-825a-4b93-a7a8-67990653b5dc","Type":"ContainerDied","Data":"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"}
Jul 14 22:31:31.716706 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.716631    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m" event={"ID":"86c7d9ce-825a-4b93-a7a8-67990653b5dc","Type":"ContainerDied","Data":"ff13b1e0c2bc8a77a022fb35c08ad9bb979159341bc960b48b699badd3b441ac"}
Jul 14 22:31:31.716706 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.716663    2570 scope.go:117] "RemoveContainer" containerID="31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"
Jul 14 22:31:31.725988 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.725967    2570 scope.go:117] "RemoveContainer" containerID="31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"
Jul 14 22:31:31.726304 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:31.726285    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f\": container with ID starting with 31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f not found: ID does not exist" containerID="31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"
Jul 14 22:31:31.726363 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.726314    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f"} err="failed to get container status \"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f\": rpc error: code = NotFound desc = could not find container \"31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f\": container with ID starting with 31e2af6b25ecaddb4dfd6f802f850f8d90c9f5e290c618458bc44db5c65c750f not found: ID does not exist"
Jul 14 22:31:31.738949 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.738908    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:31:31.742881 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.742847    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dc25fd-5cbd99645d-tdx2m"]
Jul 14 22:31:31.876520 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:31.876474    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" path="/var/lib/kubelet/pods/86c7d9ce-825a-4b93-a7a8-67990653b5dc/volumes"
Jul 14 22:31:35.690990 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:35.690955    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:42.415462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:42.415430    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:42.523459 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:42.523424    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:46.484165 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:46.484128    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:46.719135 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:46.719098    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:47.062798 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:47.062762    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:47.611891 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:47.611854    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:47.684134 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:47.684092    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.124949 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.124915    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.244938 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.244898    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:31:50.245279 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245267    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84193bca-3df3-400c-a259-0e5f0b9beef7" containerName="revoke-keys"
Jul 14 22:31:50.245279 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245280    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="84193bca-3df3-400c-a259-0e5f0b9beef7" containerName="revoke-keys"
Jul 14 22:31:50.245378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245305    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" containerName="maas-api"
Jul 14 22:31:50.245378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245311    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" containerName="maas-api"
Jul 14 22:31:50.245378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245367    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="84193bca-3df3-400c-a259-0e5f0b9beef7" containerName="revoke-keys"
Jul 14 22:31:50.245378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.245377    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c7d9ce-825a-4b93-a7a8-67990653b5dc" containerName="maas-api"
Jul 14 22:31:50.248706 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.248675    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.251965 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.251939    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-fdfd38-serving-cert\""
Jul 14 22:31:50.263945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.263915    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:31:50.301979 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.301943    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.330051 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.330015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.330267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.330086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9gq4\" (UniqueName: \"kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.330267 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.330132    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.396842 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.396732    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.430621 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.430593    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.430877 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.430656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gq4\" (UniqueName: \"kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.430877 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.430703    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.431082 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:50.431061    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-serving-cert: secret "maas-api-e2e-shared-a-fdfd38-serving-cert" not found
Jul 14 22:31:50.431236 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:50.431169    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls podName:a5632736-7a9c-4d69-abf4-f5558dec56fd nodeName:}" failed. No retries permitted until 2026-07-14 22:31:50.931147331 +0000 UTC m=+2451.619123417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls") pod "maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" (UID: "a5632736-7a9c-4d69-abf4-f5558dec56fd") : secret "maas-api-e2e-shared-a-fdfd38-serving-cert" not found
Jul 14 22:31:50.431532 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.431494    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.446513 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.446481    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9gq4\" (UniqueName: \"kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.679972 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.679860    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.899197 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.899130    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:50.936675 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.936647    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.939240 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.939215    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") pod \"maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:50.998170 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:50.998136    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:51.162901 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:51.162850    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:51.314759 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:51.314732    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:31:51.317264 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:31:51.317230    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5632736_7a9c_4d69_abf4_f5558dec56fd.slice/crio-8d256eabc7565365b6efce83d77eab69d38f71e92f1f41e2edd4924f8bdb0678 WatchSource:0}: Error finding container 8d256eabc7565365b6efce83d77eab69d38f71e92f1f41e2edd4924f8bdb0678: Status 404 returned error can't find the container with id 8d256eabc7565365b6efce83d77eab69d38f71e92f1f41e2edd4924f8bdb0678
Jul 14 22:31:51.823225 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:51.823191    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" event={"ID":"a5632736-7a9c-4d69-abf4-f5558dec56fd","Type":"ContainerStarted","Data":"8d256eabc7565365b6efce83d77eab69d38f71e92f1f41e2edd4924f8bdb0678"}
Jul 14 22:31:52.829496 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:52.829460    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" event={"ID":"a5632736-7a9c-4d69-abf4-f5558dec56fd","Type":"ContainerStarted","Data":"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"}
Jul 14 22:31:52.830065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:52.829543    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:52.845485 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:52.845428    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" podStartSLOduration=2.34526301 podStartE2EDuration="2.84540667s" totalImagesPullingTime="500.14366ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:31:52.845074108 +0000 UTC m=+2453.533050220" watchObservedRunningTime="2026-07-14 22:31:52.84540667 +0000 UTC m=+2453.533382764"
Jul 14 22:31:54.615896 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:54.615856    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:54.970368 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:54.970335    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:55.262242 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:55.262150    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:55.482251 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:55.482217    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:58.161473 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.161434    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:58.242698 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.242649    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:31:58.247207 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.247181    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.249570 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.249548    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-32d7b3-serving-cert\""
Jul 14 22:31:58.255401 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.255366    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:31:58.274148 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.274111    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:58.374546 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.374509    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:58.416675 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.416578    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlxxl\" (UniqueName: \"kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.416675 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.416645    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.416675 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.416674    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.518180 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.518141    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.518180 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.518184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.518469 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.518260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlxxl\" (UniqueName: \"kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.518469 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:58.518364    2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-serving-cert: secret "maas-api-e2e-shared-b-32d7b3-serving-cert" not found
Jul 14 22:31:58.518469 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:31:58.518437    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls podName:9ba038cd-1ba0-41b9-8035-38e4ded0a8fa nodeName:}" failed. No retries permitted until 2026-07-14 22:31:59.018419163 +0000 UTC m=+2459.706395237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls") pod "maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" (UID: "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa") : secret "maas-api-e2e-shared-b-32d7b3-serving-cert" not found
Jul 14 22:31:58.518873 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.518853    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.529325 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.529287    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlxxl\" (UniqueName: \"kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:58.829174 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.829139    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:58.840421 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.840393    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:31:58.931492 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:58.931455    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:31:59.022959 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.022920    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:59.025556 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.025524    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") pod \"maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:59.161985 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.161877    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:59.310480 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.310453    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:31:59.312857 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:31:59.312827    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba038cd_1ba0_41b9_8035_38e4ded0a8fa.slice/crio-94740830f08564f6bd1ce4e1e86c394ddfa44f74d557fc578f4808a5ecae2230 WatchSource:0}: Error finding container 94740830f08564f6bd1ce4e1e86c394ddfa44f74d557fc578f4808a5ecae2230: Status 404 returned error can't find the container with id 94740830f08564f6bd1ce4e1e86c394ddfa44f74d557fc578f4808a5ecae2230
Jul 14 22:31:59.876019 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.875981    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" event={"ID":"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa","Type":"ContainerStarted","Data":"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"}
Jul 14 22:31:59.876196 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.876034    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" event={"ID":"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa","Type":"ContainerStarted","Data":"94740830f08564f6bd1ce4e1e86c394ddfa44f74d557fc578f4808a5ecae2230"}
Jul 14 22:31:59.876196 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.876054    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:31:59.888198 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:31:59.888139    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" podStartSLOduration=1.453355892 podStartE2EDuration="1.888118451s" totalImagesPullingTime="434.762559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:31:59.886832379 +0000 UTC m=+2460.574808495" watchObservedRunningTime="2026-07-14 22:31:59.888118451 +0000 UTC m=+2460.576094545"
Jul 14 22:32:02.652772 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:02.652732    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:05.883189 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:05.883146    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:32:11.691854 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:11.691786    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:11.796161 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:11.796126    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:12.730764 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:12.730725    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:19.711275 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:19.711238    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:19.813571 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:19.813535    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:32:42.383655 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:32:42.383617    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:33:00.986795 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:33:00.986708    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:33:24.255048 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:33:24.255006    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:39.903593 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:39.903501    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:41.009699 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:41.009664    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:43.783654 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:43.783618    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:44.693378 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:44.693347    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:57.301776 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.301729    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:34:57.305460 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.305431    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.309013 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.308983    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fdfd38\"/\"kube-root-ca.crt\""
Jul 14 22:34:57.309163 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.308984    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fdfd38\"/\"default-dockercfg-r6g2p\""
Jul 14 22:34:57.309163 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.308993    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fdfd38\"/\"openshift-service-ca.crt\""
Jul 14 22:34:57.309163 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.309055    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fdfd38\"/\"sub-test-model-fdfd38-kserve-self-signed-certs\""
Jul 14 22:34:57.314906 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.314878    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:34:57.358675 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.358630    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x58xq\" (UniqueName: \"kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.359069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.359030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.359199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.359086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.359199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.359142    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.359199 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.359171    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.359383 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.359245    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.394433 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.394398    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:34:57.459987 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.459953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460046    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x58xq\" (UniqueName: \"kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460082    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460106    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460141    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460480 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460443    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460607 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.460607 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.460585    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.462549 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.462524    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.462722 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.462705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.468022 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.467994    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x58xq\" (UniqueName: \"kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq\") pod \"sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") " pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.616993 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.616892    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:34:57.768564 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.768499    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:34:57.770984 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:34:57.770954    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5e13fa3_918e_4ffa_8658_a056dbd1de5b.slice/crio-cb7f92a85ca79c4145c1016da7e6030f08ebfaa16a712bba6a614071a7df082d WatchSource:0}: Error finding container cb7f92a85ca79c4145c1016da7e6030f08ebfaa16a712bba6a614071a7df082d: Status 404 returned error can't find the container with id cb7f92a85ca79c4145c1016da7e6030f08ebfaa16a712bba6a614071a7df082d
Jul 14 22:34:57.797316 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:57.797278    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" event={"ID":"f5e13fa3-918e-4ffa-8658-a056dbd1de5b","Type":"ContainerStarted","Data":"cb7f92a85ca79c4145c1016da7e6030f08ebfaa16a712bba6a614071a7df082d"}
Jul 14 22:34:58.807585 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:58.807544    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" event={"ID":"f5e13fa3-918e-4ffa-8658-a056dbd1de5b","Type":"ContainerStarted","Data":"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"}
Jul 14 22:34:58.823661 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:34:58.823608    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" podStartSLOduration=1.823591526 podStartE2EDuration="1.823591526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:34:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:34:58.82311449 +0000 UTC m=+2639.511090595" watchObservedRunningTime="2026-07-14 22:34:58.823591526 +0000 UTC m=+2639.511567619"
Jul 14 22:35:07.617079 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:07.617036    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:35:07.617607 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:07.617093    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:35:07.630065 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:07.630033    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:35:07.866665 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:07.866629    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:35:35.534943 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:35.534904    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:35:52.842313 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:35:52.842235    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:36:00.044448 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:00.044413    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:36:00.052662 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:00.052632    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:36:00.053553 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:00.053530    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:36:00.060673 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:00.060648    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jdvlx_7d1cf725-a7b1-4a1c-989c-085b2a031405/ovn-acl-logging/0.log"
Jul 14 22:36:12.532007 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:12.531965    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:36:12.552277 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:12.552233    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:36:13.508893 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:13.508857    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:36:20.129695 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:20.129654    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:36:27.555779 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.555735    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" podUID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" containerName="main" containerID="cri-o://30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2" gracePeriod=45
Jul 14 22:36:27.812534 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.812455    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:36:27.899832 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899769    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900028 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899872    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900028 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899902    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900028 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899927    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900028 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899960    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900028 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.899993    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x58xq\" (UniqueName: \"kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq\") pod \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\" (UID: \"f5e13fa3-918e-4ffa-8658-a056dbd1de5b\") "
Jul 14 22:36:27.900304 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.900203    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:36:27.900304 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.900217    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home" (OuterVolumeSpecName: "home") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:36:27.900304 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.900227    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache" (OuterVolumeSpecName: "model-cache") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:36:27.902155 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.902123    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm" (OuterVolumeSpecName: "dshm") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 22:36:27.902288 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.902209    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:36:27.902288 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:27.902239    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq" (OuterVolumeSpecName: "kube-api-access-x58xq") pod "f5e13fa3-918e-4ffa-8658-a056dbd1de5b" (UID: "f5e13fa3-918e-4ffa-8658-a056dbd1de5b"). InnerVolumeSpecName "kube-api-access-x58xq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:36:28.000865 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000792    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tmp-dir\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.000865 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000857    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-home\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.000865 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000866    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-dshm\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.000865 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000875    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x58xq\" (UniqueName: \"kubernetes.io/projected/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-kube-api-access-x58xq\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.001147 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000884    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-tls-certs\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.001147 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.000892    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f5e13fa3-918e-4ffa-8658-a056dbd1de5b-model-cache\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:36:28.274367 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.274328    2570 generic.go:356] "Generic (PLEG): container finished" podID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" containerID="30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2" exitCode=0
Jul 14 22:36:28.274367 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.274371    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"
Jul 14 22:36:28.274580 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.274417    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" event={"ID":"f5e13fa3-918e-4ffa-8658-a056dbd1de5b","Type":"ContainerDied","Data":"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"}
Jul 14 22:36:28.274580 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.274465    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq" event={"ID":"f5e13fa3-918e-4ffa-8658-a056dbd1de5b","Type":"ContainerDied","Data":"cb7f92a85ca79c4145c1016da7e6030f08ebfaa16a712bba6a614071a7df082d"}
Jul 14 22:36:28.274580 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.274484    2570 scope.go:117] "RemoveContainer" containerID="30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"
Jul 14 22:36:28.283963 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.283944    2570 scope.go:117] "RemoveContainer" containerID="30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"
Jul 14 22:36:28.284249 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:36:28.284231    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2\": container with ID starting with 30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2 not found: ID does not exist" containerID="30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"
Jul 14 22:36:28.284308 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.284261    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2"} err="failed to get container status \"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2\": rpc error: code = NotFound desc = could not find container \"30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2\": container with ID starting with 30ed1cb4d3c05f81fce38283e6052b66de8818288c69a7b2ca94e84c2352e5a2 not found: ID does not exist"
Jul 14 22:36:28.295268 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.295238    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:36:28.298893 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:28.298865    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fdfd38/sub-test-model-fdfd38-kserve-5cddff6c49-w7sdq"]
Jul 14 22:36:29.883879 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:29.883845    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" path="/var/lib/kubelet/pods/f5e13fa3-918e-4ffa-8658-a056dbd1de5b/volumes"
Jul 14 22:36:41.019502 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:36:41.019471    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:12.439843 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:12.439790    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:12.540084 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:12.540050    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:20.570548 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:20.570511    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:20.720214 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:20.720178    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:26.598234 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:26.598204    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:27.625462 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:27.625425    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:28.571342 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.571312    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7567-139871593462576"]
Jul 14 22:37:28.571701 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.571688    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" containerName="main"
Jul 14 22:37:28.571753 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.571703    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" containerName="main"
Jul 14 22:37:28.571794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.571766    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5e13fa3-918e-4ffa-8658-a056dbd1de5b" containerName="main"
Jul 14 22:37:28.578916 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.576019    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:28.579736 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.579518    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-vx6fh\""
Jul 14 22:37:28.583147 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.583117    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593462576"]
Jul 14 22:37:28.656590 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.656545    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgvkv\" (UniqueName: \"kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv\") pod \"test-curl-7567-139871593462576\" (UID: \"ab690230-f905-45da-9182-7b874c7a0dfe\") " pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:28.757601 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.757555    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgvkv\" (UniqueName: \"kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv\") pod \"test-curl-7567-139871593462576\" (UID: \"ab690230-f905-45da-9182-7b874c7a0dfe\") " pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:28.765564 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.765534    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgvkv\" (UniqueName: \"kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv\") pod \"test-curl-7567-139871593462576\" (UID: \"ab690230-f905-45da-9182-7b874c7a0dfe\") " pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:28.891377 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:28.891270    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:29.031841 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:29.031790    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593462576"]
Jul 14 22:37:29.033912 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:37:29.033884    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab690230_f905_45da_9182_7b874c7a0dfe.slice/crio-946af68fce03e5a9083450f553be03efd09a4d5597e6b695e4ecdef17666b7e9 WatchSource:0}: Error finding container 946af68fce03e5a9083450f553be03efd09a4d5597e6b695e4ecdef17666b7e9: Status 404 returned error can't find the container with id 946af68fce03e5a9083450f553be03efd09a4d5597e6b695e4ecdef17666b7e9
Jul 14 22:37:29.035335 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:29.035314    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 22:37:29.594252 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:29.594215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593462576" event={"ID":"ab690230-f905-45da-9182-7b874c7a0dfe","Type":"ContainerStarted","Data":"946af68fce03e5a9083450f553be03efd09a4d5597e6b695e4ecdef17666b7e9"}
Jul 14 22:37:30.600304 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:30.600264    2570 generic.go:356] "Generic (PLEG): container finished" podID="ab690230-f905-45da-9182-7b874c7a0dfe" containerID="d320032901c23ac8b224b227bc0a68541d3cfbc9da24355bce66ff975defb0c1" exitCode=0
Jul 14 22:37:30.600721 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:30.600351    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593462576" event={"ID":"ab690230-f905-45da-9182-7b874c7a0dfe","Type":"ContainerDied","Data":"d320032901c23ac8b224b227bc0a68541d3cfbc9da24355bce66ff975defb0c1"}
Jul 14 22:37:31.743815 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.743773    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:31.764728 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.764688    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7567-139871593462576_ab690230-f905-45da-9182-7b874c7a0dfe/test-curl-7567-139871593462576/0.log"
Jul 14 22:37:31.788256 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.788218    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7567-139871593462576"]
Jul 14 22:37:31.792030 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.791997    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7567-139871593462576"]
Jul 14 22:37:31.889572 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.889482    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgvkv\" (UniqueName: \"kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv\") pod \"ab690230-f905-45da-9182-7b874c7a0dfe\" (UID: \"ab690230-f905-45da-9182-7b874c7a0dfe\") "
Jul 14 22:37:31.891746 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.891714    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv" (OuterVolumeSpecName: "kube-api-access-qgvkv") pod "ab690230-f905-45da-9182-7b874c7a0dfe" (UID: "ab690230-f905-45da-9182-7b874c7a0dfe"). InnerVolumeSpecName "kube-api-access-qgvkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:31.912828 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.912775    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7567-139871593462832"]
Jul 14 22:37:31.913184 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.913170    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab690230-f905-45da-9182-7b874c7a0dfe" containerName="test-curl-7567-139871593462576"
Jul 14 22:37:31.913228 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.913188    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab690230-f905-45da-9182-7b874c7a0dfe" containerName="test-curl-7567-139871593462576"
Jul 14 22:37:31.913271 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.913262    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab690230-f905-45da-9182-7b874c7a0dfe" containerName="test-curl-7567-139871593462576"
Jul 14 22:37:31.916266 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.916248    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:31.922870 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.922839    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593462832"]
Jul 14 22:37:31.991164 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:31.991131    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgvkv\" (UniqueName: \"kubernetes.io/projected/ab690230-f905-45da-9182-7b874c7a0dfe-kube-api-access-qgvkv\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:32.092351 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.092309    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsgnz\" (UniqueName: \"kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz\") pod \"test-curl-7567-139871593462832\" (UID: \"47c866e4-93a1-414e-b6f9-79f902cbce71\") " pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:32.193836 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.193789    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsgnz\" (UniqueName: \"kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz\") pod \"test-curl-7567-139871593462832\" (UID: \"47c866e4-93a1-414e-b6f9-79f902cbce71\") " pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:32.202193 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.202159    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsgnz\" (UniqueName: \"kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz\") pod \"test-curl-7567-139871593462832\" (UID: \"47c866e4-93a1-414e-b6f9-79f902cbce71\") " pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:32.227594 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.227557    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:32.365583 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.365555    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593462832"]
Jul 14 22:37:32.367483 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:37:32.367457    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47c866e4_93a1_414e_b6f9_79f902cbce71.slice/crio-04c951d78b32d61560ff258499be5c0c71400d63929273f98615a415275b0e72 WatchSource:0}: Error finding container 04c951d78b32d61560ff258499be5c0c71400d63929273f98615a415275b0e72: Status 404 returned error can't find the container with id 04c951d78b32d61560ff258499be5c0c71400d63929273f98615a415275b0e72
Jul 14 22:37:32.611292 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.611220    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462576"
Jul 14 22:37:32.611292 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.611275    2570 scope.go:117] "RemoveContainer" containerID="d320032901c23ac8b224b227bc0a68541d3cfbc9da24355bce66ff975defb0c1"
Jul 14 22:37:32.612552 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:32.612462    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593462832" event={"ID":"47c866e4-93a1-414e-b6f9-79f902cbce71","Type":"ContainerStarted","Data":"04c951d78b32d61560ff258499be5c0c71400d63929273f98615a415275b0e72"}
Jul 14 22:37:33.619234 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:33.619201    2570 generic.go:356] "Generic (PLEG): container finished" podID="47c866e4-93a1-414e-b6f9-79f902cbce71" containerID="62c62677562d33e2c70dc38ac086b43be9c086bacf25fc444af8d0354dfdc055" exitCode=0
Jul 14 22:37:33.619627 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:33.619248    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593462832" event={"ID":"47c866e4-93a1-414e-b6f9-79f902cbce71","Type":"ContainerDied","Data":"62c62677562d33e2c70dc38ac086b43be9c086bacf25fc444af8d0354dfdc055"}
Jul 14 22:37:33.875294 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:33.875213    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab690230-f905-45da-9182-7b874c7a0dfe" path="/var/lib/kubelet/pods/ab690230-f905-45da-9182-7b874c7a0dfe/volumes"
Jul 14 22:37:34.760854 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.760776    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:34.783653 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.783621    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7567-139871593462832_47c866e4-93a1-414e-b6f9-79f902cbce71/test-curl-7567-139871593462832/0.log"
Jul 14 22:37:34.807782 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.807745    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7567-139871593462832"]
Jul 14 22:37:34.811372 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.811342    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7567-139871593462832"]
Jul 14 22:37:34.919697 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.919603    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsgnz\" (UniqueName: \"kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz\") pod \"47c866e4-93a1-414e-b6f9-79f902cbce71\" (UID: \"47c866e4-93a1-414e-b6f9-79f902cbce71\") "
Jul 14 22:37:34.921746 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.921721    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz" (OuterVolumeSpecName: "kube-api-access-lsgnz") pod "47c866e4-93a1-414e-b6f9-79f902cbce71" (UID: "47c866e4-93a1-414e-b6f9-79f902cbce71"). InnerVolumeSpecName "kube-api-access-lsgnz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:34.971829 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.971771    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:34.972203 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.972186    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47c866e4-93a1-414e-b6f9-79f902cbce71" containerName="test-curl-7567-139871593462832"
Jul 14 22:37:34.972252 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.972204    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="47c866e4-93a1-414e-b6f9-79f902cbce71" containerName="test-curl-7567-139871593462832"
Jul 14 22:37:34.972287 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.972279    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="47c866e4-93a1-414e-b6f9-79f902cbce71" containerName="test-curl-7567-139871593462832"
Jul 14 22:37:34.975313 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.975294    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:34.978575 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:34.978543    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:35.020867 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.020829    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsgnz\" (UniqueName: \"kubernetes.io/projected/47c866e4-93a1-414e-b6f9-79f902cbce71-kube-api-access-lsgnz\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:35.122217 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.122180    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf4dw\" (UniqueName: \"kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw\") pod \"test-curl-7567-139871593328688\" (UID: \"1097109d-b91c-4e85-8456-d1e4efcf054b\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:35.223158 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.223116    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf4dw\" (UniqueName: \"kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw\") pod \"test-curl-7567-139871593328688\" (UID: \"1097109d-b91c-4e85-8456-d1e4efcf054b\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:35.232295 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.232266    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf4dw\" (UniqueName: \"kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw\") pod \"test-curl-7567-139871593328688\" (UID: \"1097109d-b91c-4e85-8456-d1e4efcf054b\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:35.287470 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.287434    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:35.424565 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.424447    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:35.426838 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:37:35.426795    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1097109d_b91c_4e85_8456_d1e4efcf054b.slice/crio-7559960c85737aed21180c387dfb4206a3bb3182029eda63dac238005777e439 WatchSource:0}: Error finding container 7559960c85737aed21180c387dfb4206a3bb3182029eda63dac238005777e439: Status 404 returned error can't find the container with id 7559960c85737aed21180c387dfb4206a3bb3182029eda63dac238005777e439
Jul 14 22:37:35.630799 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.630711    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:35.630984 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.630713    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04c951d78b32d61560ff258499be5c0c71400d63929273f98615a415275b0e72"
Jul 14 22:37:35.632089 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.632050    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593328688" event={"ID":"1097109d-b91c-4e85-8456-d1e4efcf054b","Type":"ContainerStarted","Data":"7559960c85737aed21180c387dfb4206a3bb3182029eda63dac238005777e439"}
Jul 14 22:37:35.642462 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:37:35.642428    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7567-139871593462832\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="47c866e4-93a1-414e-b6f9-79f902cbce71" pod="opendatahub/test-curl-7567-139871593462832"
Jul 14 22:37:35.875737 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:35.875700    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47c866e4-93a1-414e-b6f9-79f902cbce71" path="/var/lib/kubelet/pods/47c866e4-93a1-414e-b6f9-79f902cbce71/volumes"
Jul 14 22:37:36.639225 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:36.639189    2570 generic.go:356] "Generic (PLEG): container finished" podID="1097109d-b91c-4e85-8456-d1e4efcf054b" containerID="408d84ce60f62fc836061d95a7772502e0832bbbd2039340bbd368fe5ea1208d" exitCode=0
Jul 14 22:37:36.639438 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:36.639239    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593328688" event={"ID":"1097109d-b91c-4e85-8456-d1e4efcf054b","Type":"ContainerDied","Data":"408d84ce60f62fc836061d95a7772502e0832bbbd2039340bbd368fe5ea1208d"}
Jul 14 22:37:37.786494 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.786464    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:37.816463 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.816436    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7567-139871593328688_1097109d-b91c-4e85-8456-d1e4efcf054b/test-curl-7567-139871593328688/0.log"
Jul 14 22:37:37.842559 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.842510    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:37.845586 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.845554    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:37.845767 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.845744    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sf4dw\" (UniqueName: \"kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw\") pod \"1097109d-b91c-4e85-8456-d1e4efcf054b\" (UID: \"1097109d-b91c-4e85-8456-d1e4efcf054b\") "
Jul 14 22:37:37.847911 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.847885    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw" (OuterVolumeSpecName: "kube-api-access-sf4dw") pod "1097109d-b91c-4e85-8456-d1e4efcf054b" (UID: "1097109d-b91c-4e85-8456-d1e4efcf054b"). InnerVolumeSpecName "kube-api-access-sf4dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:37.875337 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.875299    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1097109d-b91c-4e85-8456-d1e4efcf054b" path="/var/lib/kubelet/pods/1097109d-b91c-4e85-8456-d1e4efcf054b/volumes"
Jul 14 22:37:37.946444 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.946406    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sf4dw\" (UniqueName: \"kubernetes.io/projected/1097109d-b91c-4e85-8456-d1e4efcf054b-kube-api-access-sf4dw\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:37.966715 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.966681    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:37.967110 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.967095    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1097109d-b91c-4e85-8456-d1e4efcf054b" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:37.967153 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.967111    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1097109d-b91c-4e85-8456-d1e4efcf054b" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:37.967193 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.967184    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1097109d-b91c-4e85-8456-d1e4efcf054b" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:37.970122 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.970101    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:37.976346 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:37.976309    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:38.047526 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.047483    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnsg7\" (UniqueName: \"kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7\") pod \"test-curl-7567-139871593328688\" (UID: \"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:38.148478 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.148443    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnsg7\" (UniqueName: \"kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7\") pod \"test-curl-7567-139871593328688\" (UID: \"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:38.157862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.157828    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnsg7\" (UniqueName: \"kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7\") pod \"test-curl-7567-139871593328688\" (UID: \"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e\") " pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:38.281005 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.280907    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:38.426873 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.426726    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:38.430534 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:37:38.430489    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf576aaf7_c2c5_40ff_8b6f_03edefc9d72e.slice/crio-fff8fd90696ec22ccdfba2faf25c996760238a3aff188a711482843eb577421f WatchSource:0}: Error finding container fff8fd90696ec22ccdfba2faf25c996760238a3aff188a711482843eb577421f: Status 404 returned error can't find the container with id fff8fd90696ec22ccdfba2faf25c996760238a3aff188a711482843eb577421f
Jul 14 22:37:38.650383 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.650290    2570 scope.go:117] "RemoveContainer" containerID="408d84ce60f62fc836061d95a7772502e0832bbbd2039340bbd368fe5ea1208d"
Jul 14 22:37:38.650383 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.650291    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:38.651739 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:38.651622    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593328688" event={"ID":"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e","Type":"ContainerStarted","Data":"fff8fd90696ec22ccdfba2faf25c996760238a3aff188a711482843eb577421f"}
Jul 14 22:37:39.020333 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:39.020303    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:39.659603 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:39.659565    2570 generic.go:356] "Generic (PLEG): container finished" podID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" containerID="ffaff5aa963332c41667c6352b404fed08039c7864a6eea64f8b88d96411e436" exitCode=0
Jul 14 22:37:39.659821 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:39.659662    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593328688" event={"ID":"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e","Type":"ContainerDied","Data":"ffaff5aa963332c41667c6352b404fed08039c7864a6eea64f8b88d96411e436"}
Jul 14 22:37:40.814367 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.814332    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:40.834318 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.834288    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7567-139871593328688_f576aaf7-c2c5-40ff-8b6f-03edefc9d72e/test-curl-7567-139871593328688/0.log"
Jul 14 22:37:40.859245 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.859163    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:40.861584 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.861554    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7567-139871593328688"]
Jul 14 22:37:40.873938 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.873903    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mnsg7\" (UniqueName: \"kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7\") pod \"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e\" (UID: \"f576aaf7-c2c5-40ff-8b6f-03edefc9d72e\") "
Jul 14 22:37:40.876137 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.876102    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7" (OuterVolumeSpecName: "kube-api-access-mnsg7") pod "f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" (UID: "f576aaf7-c2c5-40ff-8b6f-03edefc9d72e"). InnerVolumeSpecName "kube-api-access-mnsg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:40.975037 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.975000    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnsg7\" (UniqueName: \"kubernetes.io/projected/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e-kube-api-access-mnsg7\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:40.985146 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.985106    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7567-139871593329696"]
Jul 14 22:37:40.985504 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.985490    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:40.985549 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.985505    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:40.985604 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.985595    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" containerName="test-curl-7567-139871593328688"
Jul 14 22:37:40.988643 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.988620    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:40.992523 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:40.992493    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593329696"]
Jul 14 22:37:41.076369 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.076326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9xxw\" (UniqueName: \"kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw\") pod \"test-curl-7567-139871593329696\" (UID: \"f449a555-d540-4900-b161-f4cbf70c3356\") " pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:41.177182 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.177141    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9xxw\" (UniqueName: \"kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw\") pod \"test-curl-7567-139871593329696\" (UID: \"f449a555-d540-4900-b161-f4cbf70c3356\") " pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:41.186039 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.186002    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9xxw\" (UniqueName: \"kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw\") pod \"test-curl-7567-139871593329696\" (UID: \"f449a555-d540-4900-b161-f4cbf70c3356\") " pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:41.300580 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.300483    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:41.441892 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.441862    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7567-139871593329696"]
Jul 14 22:37:41.443466 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:37:41.443440    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf449a555_d540_4900_b161_f4cbf70c3356.slice/crio-e3b438f052b1cd5ba761bd6065e1e2736b04662880c0c6d86f8487b7a09a4f69 WatchSource:0}: Error finding container e3b438f052b1cd5ba761bd6065e1e2736b04662880c0c6d86f8487b7a09a4f69: Status 404 returned error can't find the container with id e3b438f052b1cd5ba761bd6065e1e2736b04662880c0c6d86f8487b7a09a4f69
Jul 14 22:37:41.671340 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.671246    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:41.671340 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.671287    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fff8fd90696ec22ccdfba2faf25c996760238a3aff188a711482843eb577421f"
Jul 14 22:37:41.672480 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.672455    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593329696" event={"ID":"f449a555-d540-4900-b161-f4cbf70c3356","Type":"ContainerStarted","Data":"e3b438f052b1cd5ba761bd6065e1e2736b04662880c0c6d86f8487b7a09a4f69"}
Jul 14 22:37:41.681687 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:37:41.681653    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7567-139871593328688\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" pod="opendatahub/test-curl-7567-139871593328688"
Jul 14 22:37:41.876342 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:41.876303    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f576aaf7-c2c5-40ff-8b6f-03edefc9d72e" path="/var/lib/kubelet/pods/f576aaf7-c2c5-40ff-8b6f-03edefc9d72e/volumes"
Jul 14 22:37:42.679219 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:42.679182    2570 generic.go:356] "Generic (PLEG): container finished" podID="f449a555-d540-4900-b161-f4cbf70c3356" containerID="47b3deeb8c4b98451463f1ff161480b9f7bf25b90b4fcdc85e294d8e0b14575b" exitCode=0
Jul 14 22:37:42.679414 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:42.679273    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7567-139871593329696" event={"ID":"f449a555-d540-4900-b161-f4cbf70c3356","Type":"ContainerDied","Data":"47b3deeb8c4b98451463f1ff161480b9f7bf25b90b4fcdc85e294d8e0b14575b"}
Jul 14 22:37:43.829017 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.828989    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:43.847560 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.847529    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7567-139871593329696_f449a555-d540-4900-b161-f4cbf70c3356/test-curl-7567-139871593329696/0.log"
Jul 14 22:37:43.879977 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.879950    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7567-139871593329696"]
Jul 14 22:37:43.880140 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.879996    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7567-139871593329696"]
Jul 14 22:37:43.901343 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.901311    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9xxw\" (UniqueName: \"kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw\") pod \"f449a555-d540-4900-b161-f4cbf70c3356\" (UID: \"f449a555-d540-4900-b161-f4cbf70c3356\") "
Jul 14 22:37:43.903526 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:43.903498    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw" (OuterVolumeSpecName: "kube-api-access-f9xxw") pod "f449a555-d540-4900-b161-f4cbf70c3356" (UID: "f449a555-d540-4900-b161-f4cbf70c3356"). InnerVolumeSpecName "kube-api-access-f9xxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:44.003011 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:44.002969    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9xxw\" (UniqueName: \"kubernetes.io/projected/f449a555-d540-4900-b161-f4cbf70c3356-kube-api-access-f9xxw\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:44.690618 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:44.690589    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:44.690823 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:44.690598    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3b438f052b1cd5ba761bd6065e1e2736b04662880c0c6d86f8487b7a09a4f69"
Jul 14 22:37:44.700365 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:37:44.700331    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7567-139871593329696\" is forbidden: User \"system:node:ip-10-0-130-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-163.ec2.internal' and this object" podUID="f449a555-d540-4900-b161-f4cbf70c3356" pod="opendatahub/test-curl-7567-139871593329696"
Jul 14 22:37:45.876219 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:45.876186    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f449a555-d540-4900-b161-f4cbf70c3356" path="/var/lib/kubelet/pods/f449a555-d540-4900-b161-f4cbf70c3356/volumes"
Jul 14 22:37:47.610507 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:47.610469    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:47.711384 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:47.711349    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:57.335171 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.335133    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:37:57.336061 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.336002    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" podUID="a5632736-7a9c-4d69-abf4-f5558dec56fd" containerName="maas-api" containerID="cri-o://422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a" gracePeriod=30
Jul 14 22:37:57.363268 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.363234    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:37:57.596063 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.595983    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:37:57.731295 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.731260    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca\") pod \"a5632736-7a9c-4d69-abf4-f5558dec56fd\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") "
Jul 14 22:37:57.731295 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.731296    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9gq4\" (UniqueName: \"kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4\") pod \"a5632736-7a9c-4d69-abf4-f5558dec56fd\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") "
Jul 14 22:37:57.731564 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.731339    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") pod \"a5632736-7a9c-4d69-abf4-f5558dec56fd\" (UID: \"a5632736-7a9c-4d69-abf4-f5558dec56fd\") "
Jul 14 22:37:57.731739 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.731709    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5632736-7a9c-4d69-abf4-f5558dec56fd" (UID: "a5632736-7a9c-4d69-abf4-f5558dec56fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:37:57.733719 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.733680    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5632736-7a9c-4d69-abf4-f5558dec56fd" (UID: "a5632736-7a9c-4d69-abf4-f5558dec56fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:37:57.733866 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.733732    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4" (OuterVolumeSpecName: "kube-api-access-n9gq4") pod "a5632736-7a9c-4d69-abf4-f5558dec56fd" (UID: "a5632736-7a9c-4d69-abf4-f5558dec56fd"). InnerVolumeSpecName "kube-api-access-n9gq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:37:57.762615 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.762573    2570 generic.go:356] "Generic (PLEG): container finished" podID="a5632736-7a9c-4d69-abf4-f5558dec56fd" containerID="422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a" exitCode=0
Jul 14 22:37:57.762794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.762628    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"
Jul 14 22:37:57.762794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.762663    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" event={"ID":"a5632736-7a9c-4d69-abf4-f5558dec56fd","Type":"ContainerDied","Data":"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"}
Jul 14 22:37:57.762794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.762726    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk" event={"ID":"a5632736-7a9c-4d69-abf4-f5558dec56fd","Type":"ContainerDied","Data":"8d256eabc7565365b6efce83d77eab69d38f71e92f1f41e2edd4924f8bdb0678"}
Jul 14 22:37:57.762794 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.762749    2570 scope.go:117] "RemoveContainer" containerID="422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"
Jul 14 22:37:57.772640 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.772623    2570 scope.go:117] "RemoveContainer" containerID="422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"
Jul 14 22:37:57.772984 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:37:57.772962    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a\": container with ID starting with 422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a not found: ID does not exist" containerID="422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"
Jul 14 22:37:57.773048 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.772993    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a"} err="failed to get container status \"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a\": rpc error: code = NotFound desc = could not find container \"422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a\": container with ID starting with 422263cc1ea77dd11d30919f96cc730cae86103cc5239203b345fe4f07e3378a not found: ID does not exist"
Jul 14 22:37:57.783472 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.783437    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:37:57.787434 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.787401    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-fdfd38-54c7b756c-6wmmk"]
Jul 14 22:37:57.832923 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.832882    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5632736-7a9c-4d69-abf4-f5558dec56fd-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:57.832923 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.832923    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9gq4\" (UniqueName: \"kubernetes.io/projected/a5632736-7a9c-4d69-abf4-f5558dec56fd-kube-api-access-n9gq4\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:57.833168 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.832941    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5632736-7a9c-4d69-abf4-f5558dec56fd-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:37:57.875504 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:37:57.875419    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5632736-7a9c-4d69-abf4-f5558dec56fd" path="/var/lib/kubelet/pods/a5632736-7a9c-4d69-abf4-f5558dec56fd/volumes"
Jul 14 22:38:02.022485 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:02.022447    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:07.824018 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:07.823984    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:07.924099 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:07.924049    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:08.662835 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:08.662745    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:12.717519 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:12.717486    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:12.954137 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:12.954102    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:13.072859 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.072748    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"]
Jul 14 22:38:13.073172 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073155    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f449a555-d540-4900-b161-f4cbf70c3356" containerName="test-curl-7567-139871593329696"
Jul 14 22:38:13.073232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073174    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f449a555-d540-4900-b161-f4cbf70c3356" containerName="test-curl-7567-139871593329696"
Jul 14 22:38:13.073232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073195    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5632736-7a9c-4d69-abf4-f5558dec56fd" containerName="maas-api"
Jul 14 22:38:13.073232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073200    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5632736-7a9c-4d69-abf4-f5558dec56fd" containerName="maas-api"
Jul 14 22:38:13.073326 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073256    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f449a555-d540-4900-b161-f4cbf70c3356" containerName="test-curl-7567-139871593329696"
Jul 14 22:38:13.073326 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.073267    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5632736-7a9c-4d69-abf4-f5558dec56fd" containerName="maas-api"
Jul 14 22:38:13.081603 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.081573    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:13.083498 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.083468    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"]
Jul 14 22:38:13.084147 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.084113    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-4jlcq\""
Jul 14 22:38:13.180909 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.180866    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h\" (UID: \"38fecb71-0ad1-42bc-8825-b07a164f0d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:13.281412 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.281373    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h\" (UID: \"38fecb71-0ad1-42bc-8825-b07a164f0d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:13.282089 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.282066    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h\" (UID: \"38fecb71-0ad1-42bc-8825-b07a164f0d15\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:13.393600 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.393494    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:13.552880 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.552846    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"]
Jul 14 22:38:13.555362 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:38:13.555335    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38fecb71_0ad1_42bc_8825_b07a164f0d15.slice/crio-64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e WatchSource:0}: Error finding container 64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e: Status 404 returned error can't find the container with id 64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e
Jul 14 22:38:13.714884 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.714840    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:13.817336 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.817307    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:13.857704 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.857670    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h" event={"ID":"38fecb71-0ad1-42bc-8825-b07a164f0d15","Type":"ContainerStarted","Data":"ded39ef55120720543156ed0050d70009f7128b6c96d5ddc333bfc75bf97814a"}
Jul 14 22:38:13.857916 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.857724    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h" event={"ID":"38fecb71-0ad1-42bc-8825-b07a164f0d15","Type":"ContainerStarted","Data":"64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e"}
Jul 14 22:38:13.874501 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:13.874440    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h" podStartSLOduration=0.874416394 podStartE2EDuration="874.416394ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:38:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:38:13.872015433 +0000 UTC m=+2834.559991578" watchObservedRunningTime="2026-07-14 22:38:13.874416394 +0000 UTC m=+2834.562392488"
Jul 14 22:38:14.864592 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:14.864550    2570 generic.go:356] "Generic (PLEG): container finished" podID="38fecb71-0ad1-42bc-8825-b07a164f0d15" containerID="ded39ef55120720543156ed0050d70009f7128b6c96d5ddc333bfc75bf97814a" exitCode=0
Jul 14 22:38:14.865000 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:14.864599    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h" event={"ID":"38fecb71-0ad1-42bc-8825-b07a164f0d15","Type":"ContainerDied","Data":"ded39ef55120720543156ed0050d70009f7128b6c96d5ddc333bfc75bf97814a"}
Jul 14 22:38:16.024634 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.024607    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:16.102712 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.102672    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca\") pod \"38fecb71-0ad1-42bc-8825-b07a164f0d15\" (UID: \"38fecb71-0ad1-42bc-8825-b07a164f0d15\") "
Jul 14 22:38:16.103074 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.103052    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "38fecb71-0ad1-42bc-8825-b07a164f0d15" (UID: "38fecb71-0ad1-42bc-8825-b07a164f0d15"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:38:16.204154 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.204116    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/38fecb71-0ad1-42bc-8825-b07a164f0d15-maas-api-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:38:16.875783 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.875750    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h"
Jul 14 22:38:16.875988 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.875783    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h" event={"ID":"38fecb71-0ad1-42bc-8825-b07a164f0d15","Type":"ContainerDied","Data":"64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e"}
Jul 14 22:38:16.875988 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:16.875852    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64cd1f09d06ec09d982cf5bbb44084005e1d1ee1cef5c276124f3cd127ac6c2e"
Jul 14 22:38:23.480412 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.480369    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:38:23.480904 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.480640    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" podUID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" containerName="maas-api" containerID="cri-o://c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7" gracePeriod=30
Jul 14 22:38:23.502542 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.502513    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:23.747097 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.747017    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:38:23.768483 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.768450    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca\") pod \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") "
Jul 14 22:38:23.768667 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.768503    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlxxl\" (UniqueName: \"kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl\") pod \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") "
Jul 14 22:38:23.768667 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.768584    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") pod \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\" (UID: \"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa\") "
Jul 14 22:38:23.768943 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.768915    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" (UID: "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 22:38:23.770788 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.770756    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" (UID: "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 22:38:23.770964 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.770880    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl" (OuterVolumeSpecName: "kube-api-access-nlxxl") pod "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" (UID: "9ba038cd-1ba0-41b9-8035-38e4ded0a8fa"). InnerVolumeSpecName "kube-api-access-nlxxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 22:38:23.870231 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.870184    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-maas-api-tls\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:38:23.870231 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.870227    2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-openshift-service-ca\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:38:23.870441 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.870243    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlxxl\" (UniqueName: \"kubernetes.io/projected/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa-kube-api-access-nlxxl\") on node \"ip-10-0-130-163.ec2.internal\" DevicePath \"\""
Jul 14 22:38:23.915581 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.915537    2570 generic.go:356] "Generic (PLEG): container finished" podID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" containerID="c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7" exitCode=0
Jul 14 22:38:23.915760 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.915606    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"
Jul 14 22:38:23.915760 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.915630    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" event={"ID":"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa","Type":"ContainerDied","Data":"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"}
Jul 14 22:38:23.915760 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.915709    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj" event={"ID":"9ba038cd-1ba0-41b9-8035-38e4ded0a8fa","Type":"ContainerDied","Data":"94740830f08564f6bd1ce4e1e86c394ddfa44f74d557fc578f4808a5ecae2230"}
Jul 14 22:38:23.915760 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.915732    2570 scope.go:117] "RemoveContainer" containerID="c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"
Jul 14 22:38:23.925225 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.925204    2570 scope.go:117] "RemoveContainer" containerID="c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"
Jul 14 22:38:23.925562 ip-10-0-130-163 kubenswrapper[2570]: E0714 22:38:23.925540    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7\": container with ID starting with c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7 not found: ID does not exist" containerID="c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"
Jul 14 22:38:23.925611 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.925573    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7"} err="failed to get container status \"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7\": rpc error: code = NotFound desc = could not find container \"c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7\": container with ID starting with c3d65c1188074a8ee9e5bb9eae8722f3d4222abfb2aa83788112a4b29fef67b7 not found: ID does not exist"
Jul 14 22:38:23.933018 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.932982    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:38:23.935069 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:23.935038    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-32d7b3-7f756db5b4-8ckrj"]
Jul 14 22:38:25.875767 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:25.875736    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" path="/var/lib/kubelet/pods/9ba038cd-1ba0-41b9-8035-38e4ded0a8fa/volumes"
Jul 14 22:38:28.194385 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:28.194351    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:34.166401 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:34.166360    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:34.267113 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:34.267074    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:38.865332 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:38.865295    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:39.087281 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:39.087248    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:44.305574 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:44.305522    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-269c5_4b9faa79-d47a-4a43-97ea-27175eaecfc3/manager/0.log"
Jul 14 22:38:44.422865 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:44.422829    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zqm5f_62f669f2-7166-41c7-b4f7-fef4275cbfdd/manager/0.log"
Jul 14 22:38:45.010506 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:45.010477    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c99649887-j8pwd_fea1c75a-2dbd-4fe5-8520-2c4de6fa004a/copy-manifests/0.log"
Jul 14 22:38:45.017991 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:45.017965    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c99649887-j8pwd_fea1c75a-2dbd-4fe5-8520-2c4de6fa004a/manager/0.log"
Jul 14 22:38:45.561561 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:45.561529    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8559b688dd-sh5pc_7ff39070-325b-4447-972c-39af86c78078/maas-api/0.log"
Jul 14 22:38:45.810242 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:45.810213    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29734470-npnhk_1c6f8298-9032-4a2a-b458-fbe5a2316b04/cleanup/0.log"
Jul 14 22:38:46.274637 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:46.274605    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-dc25fd-f5grd_84193bca-3df3-400c-a259-0e5f0b9beef7/revoke-keys/0.log"
Jul 14 22:38:46.603126 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:46.603041    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-b4e43e-qw8cx_706c9a81-9bb9-4d05-a269-90af42f6ed65/revoke-keys/0.log"
Jul 14 22:38:47.195193 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:47.195158    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-32d7b3-h8k2h_38fecb71-0ad1-42bc-8825-b07a164f0d15/revoke-keys/0.log"
Jul 14 22:38:47.312020 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:47.311991    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p6s42_a862e6c1-b00a-421d-b88b-a35940940e84/postgres/0.log"
Jul 14 22:38:48.705517 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:48.705488    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4hcsb_f0f332b6-2b68-4e8f-89eb-92825590ad06/manager/0.log"
Jul 14 22:38:48.815715 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:48.815658    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9xn5m_1f9380f5-87b8-49d7-b119-4d8232929d41/manager/0.log"
Jul 14 22:38:48.924171 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:48.924144    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2vm85_c980cea2-7a18-437e-b8a6-ef5f4bf5f996/kuadrant-console-plugin/0.log"
Jul 14 22:38:49.048323 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:49.048239    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-brfqp_12312a97-d4bf-4755-8fad-6cf665811b32/registry-server/0.log"
Jul 14 22:38:49.172711 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:49.172682    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c6q94_40df9a6b-14e3-4e89-a223-02388737a0cd/manager/0.log"
Jul 14 22:38:49.401821 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:49.401715    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vx9bm_92e28a90-22cc-491b-8292-436a447e4c3b/manager/0.log"
Jul 14 22:38:49.995624 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:49.995591    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dcf98947-d5nvn_e4db6d7b-83b1-4af9-9684-a5d87c74d924/kube-auth-proxy/0.log"
Jul 14 22:38:50.347686 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:50.347588    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-kcssg_294cb6a0-fcdf-4a18-b372-7cf2f48ba629/payload-pre-processing/0.log"
Jul 14 22:38:51.040086 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:51.040001    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hlm4c_8b09fa8a-809e-4d5d-8e0b-f6233f19077b/main/0.log"
Jul 14 22:38:51.273126 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:51.273097    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dblrqmq_876b5a27-e6cf-43e0-b701-8289bf394abc/main/0.log"
Jul 14 22:38:51.383374 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:51.383292    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-pp687_09fcb1d8-8e11-414a-bd1b-4701754a3b38/main/0.log"
Jul 14 22:38:55.978048 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978008    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tpwxp/must-gather-xrpg6"]
Jul 14 22:38:55.978442 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978372    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" containerName="maas-api"
Jul 14 22:38:55.978442 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978382    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" containerName="maas-api"
Jul 14 22:38:55.978442 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978398    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38fecb71-0ad1-42bc-8825-b07a164f0d15" containerName="revoke-keys"
Jul 14 22:38:55.978442 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978404    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="38fecb71-0ad1-42bc-8825-b07a164f0d15" containerName="revoke-keys"
Jul 14 22:38:55.978577 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978462    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ba038cd-1ba0-41b9-8035-38e4ded0a8fa" containerName="maas-api"
Jul 14 22:38:55.978577 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.978469    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="38fecb71-0ad1-42bc-8825-b07a164f0d15" containerName="revoke-keys"
Jul 14 22:38:55.981945 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.981923    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:55.984672 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.984651    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpwxp\"/\"openshift-service-ca.crt\""
Jul 14 22:38:55.985553 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.985535    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tpwxp\"/\"kube-root-ca.crt\""
Jul 14 22:38:55.985647 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.985540    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tpwxp\"/\"default-dockercfg-m2k6m\""
Jul 14 22:38:55.986676 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.986649    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:55.991939 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:55.991919    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpwxp/must-gather-xrpg6"]
Jul 14 22:38:56.050604 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.050565    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:38:56.054677 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.054653    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1251e10a-2d07-4ac6-8072-da326065afd5-must-gather-output\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.054769 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.054696    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd5m5\" (UniqueName: \"kubernetes.io/projected/1251e10a-2d07-4ac6-8072-da326065afd5-kube-api-access-pd5m5\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.155662 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.155619    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd5m5\" (UniqueName: \"kubernetes.io/projected/1251e10a-2d07-4ac6-8072-da326065afd5-kube-api-access-pd5m5\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.155886 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.155705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1251e10a-2d07-4ac6-8072-da326065afd5-must-gather-output\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.156031 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.156015    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1251e10a-2d07-4ac6-8072-da326065afd5-must-gather-output\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.164091 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.164065    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd5m5\" (UniqueName: \"kubernetes.io/projected/1251e10a-2d07-4ac6-8072-da326065afd5-kube-api-access-pd5m5\") pod \"must-gather-xrpg6\" (UID: \"1251e10a-2d07-4ac6-8072-da326065afd5\") " pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.292076 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.291969    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpwxp/must-gather-xrpg6"
Jul 14 22:38:56.448557 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:56.448525    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpwxp/must-gather-xrpg6"]
Jul 14 22:38:57.089887 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:57.089844    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpwxp/must-gather-xrpg6" event={"ID":"1251e10a-2d07-4ac6-8072-da326065afd5","Type":"ContainerStarted","Data":"88b21619872a140f5bd9ea25f41497f2999c43bef14d0e0d30aea9ac6dc4e1a8"}
Jul 14 22:38:58.099416 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:58.099377    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpwxp/must-gather-xrpg6" event={"ID":"1251e10a-2d07-4ac6-8072-da326065afd5","Type":"ContainerStarted","Data":"4083609a406d66e1371d20a090cb2d2661c154674e2860355f3d63692c3ea6ee"}
Jul 14 22:38:58.099848 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:58.099427    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpwxp/must-gather-xrpg6" event={"ID":"1251e10a-2d07-4ac6-8072-da326065afd5","Type":"ContainerStarted","Data":"6814aecd7998ff8bac0cd48d648aec8ed11220b14b1539b3f266c271c92788b9"}
Jul 14 22:38:58.116062 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:58.116004    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tpwxp/must-gather-xrpg6" podStartSLOduration=2.360282258 podStartE2EDuration="3.115987488s" totalImagesPullingTime="755.70523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 22:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:38:58.113271256 +0000 UTC m=+2878.801247351" watchObservedRunningTime="2026-07-14 22:38:58.115987488 +0000 UTC m=+2878.803963580"
Jul 14 22:38:58.859694 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:58.859657    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-55sxh_385bbb00-e382-4a0b-927b-13a51ee2d1d3/global-pull-secret-syncer/0.log"
Jul 14 22:38:58.953380 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:58.953345    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log"
Jul 14 22:38:59.038116 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:38:59.038081    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-163.ec2.internal_d13ebaf0952296d4b090313385ef7786/haproxy/0.log"
Jul 14 22:39:04.830041 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:04.829993    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4hcsb_f0f332b6-2b68-4e8f-89eb-92825590ad06/manager/0.log"
Jul 14 22:39:04.871459 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:04.871419    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-9xn5m_1f9380f5-87b8-49d7-b119-4d8232929d41/manager/0.log"
Jul 14 22:39:04.917272 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:04.917246    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2vm85_c980cea2-7a18-437e-b8a6-ef5f4bf5f996/kuadrant-console-plugin/0.log"
Jul 14 22:39:04.974010 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:04.973973    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-brfqp_12312a97-d4bf-4755-8fad-6cf665811b32/registry-server/0.log"
Jul 14 22:39:05.122881 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:05.121026    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-c6q94_40df9a6b-14e3-4e89-a223-02388737a0cd/manager/0.log"
Jul 14 22:39:05.239958 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:05.239922    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vx9bm_92e28a90-22cc-491b-8292-436a447e4c3b/manager/0.log"
Jul 14 22:39:07.442506 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:07.442471    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6j6p_b4bcb077-6179-4b3a-b247-5faf6ef426a4/node-exporter/0.log"
Jul 14 22:39:07.467635 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:07.467593    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6j6p_b4bcb077-6179-4b3a-b247-5faf6ef426a4/kube-rbac-proxy/0.log"
Jul 14 22:39:07.494280 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:07.494253    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d6j6p_b4bcb077-6179-4b3a-b247-5faf6ef426a4/init-textfile/0.log"
Jul 14 22:39:09.202313 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.202267    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"]
Jul 14 22:39:09.209131 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.209094    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.214606 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.214566    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"]
Jul 14 22:39:09.316652 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.316602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-podres\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.316874 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.316682    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkhrw\" (UniqueName: \"kubernetes.io/projected/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-kube-api-access-tkhrw\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.316874 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.316730    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-sys\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.316874 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.316769    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-lib-modules\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.317108 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.316883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-proc\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417565 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkhrw\" (UniqueName: \"kubernetes.io/projected/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-kube-api-access-tkhrw\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417565 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-sys\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-lib-modules\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417650    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-proc\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417673    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-podres\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417728    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-sys\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417792    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-podres\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417795    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-proc\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.417862 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.417825    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-lib-modules\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.432434 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.432393    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkhrw\" (UniqueName: \"kubernetes.io/projected/40ee0b84-0675-4c5d-adfe-89b3b6cb65c2-kube-api-access-tkhrw\") pod \"perf-node-gather-daemonset-v4lhf\" (UID: \"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2\") " pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.526785 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.526689    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:09.722663 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:09.722609    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"]
Jul 14 22:39:09.729321 ip-10-0-130-163 kubenswrapper[2570]: W0714 22:39:09.729289    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod40ee0b84_0675_4c5d_adfe_89b3b6cb65c2.slice/crio-a14df363e38a68752456e8bd8b906314de6fe2f2e00124afa570a406bafe6277 WatchSource:0}: Error finding container a14df363e38a68752456e8bd8b906314de6fe2f2e00124afa570a406bafe6277: Status 404 returned error can't find the container with id a14df363e38a68752456e8bd8b906314de6fe2f2e00124afa570a406bafe6277
Jul 14 22:39:10.039557 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.039479    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/1.log"
Jul 14 22:39:10.044576 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.044536    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-6sqmw_e515f36b-7029-42ae-b953-4220a93b734e/console-operator/2.log"
Jul 14 22:39:10.197981 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.197939    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf" event={"ID":"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2","Type":"ContainerStarted","Data":"32f445a8cbee76d7d901ac593045da2ae459ae569eca23bc220c03c8cf20f69c"}
Jul 14 22:39:10.198160 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.197996    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf" event={"ID":"40ee0b84-0675-4c5d-adfe-89b3b6cb65c2","Type":"ContainerStarted","Data":"a14df363e38a68752456e8bd8b906314de6fe2f2e00124afa570a406bafe6277"}
Jul 14 22:39:10.198160 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.198091    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:10.217704 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:10.217652    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf" podStartSLOduration=1.217635446 podStartE2EDuration="1.217635446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 22:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 22:39:10.215433181 +0000 UTC m=+2890.903409289" watchObservedRunningTime="2026-07-14 22:39:10.217635446 +0000 UTC m=+2890.905611538"
Jul 14 22:39:12.399548 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:12.399517    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qgrtx_f2ff2305-b9cc-46d7-b766-2d3697a1e7cc/dns/0.log"
Jul 14 22:39:12.427281 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:12.427243    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qgrtx_f2ff2305-b9cc-46d7-b766-2d3697a1e7cc/kube-rbac-proxy/0.log"
Jul 14 22:39:12.478279 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:12.478253    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hnntz_7ccf0eb0-d809-494f-9a00-998e2c04af26/dns-node-resolver/0.log"
Jul 14 22:39:13.083908 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:13.083869    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-75f9c687fc-2cf7f_fdcf2fe2-f57d-4382-9dcf-bd603ec06030/registry/0.log"
Jul 14 22:39:13.106700 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:13.106669    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z86g_19a004c4-cced-4841-9e55-bee9abe6d81c/node-ca/0.log"
Jul 14 22:39:14.338636 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:14.338608    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dcf98947-d5nvn_e4db6d7b-83b1-4af9-9684-a5d87c74d924/kube-auth-proxy/0.log"
Jul 14 22:39:14.463035 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:14.462998    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-kcssg_294cb6a0-fcdf-4a18-b372-7cf2f48ba629/payload-pre-processing/0.log"
Jul 14 22:39:14.842816 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:14.842770    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:39:14.898540 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:14.898499    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-87xhf_38e2c79d-8955-4bfe-b7ff-c671bff1a118/konnectivity-agent/0.log" limit=1024
Jul 14 22:39:15.292956 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:15.292924    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tlh5c_322680e0-990d-495b-98d4-70629064e077/serve-healthcheck-canary/0.log"
Jul 14 22:39:15.855376 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:15.855337    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lght6_8d788a96-bf9b-4958-b4c5-d0ef6893b7cf/kube-rbac-proxy/0.log"
Jul 14 22:39:15.877881 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:15.877857    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lght6_8d788a96-bf9b-4958-b4c5-d0ef6893b7cf/exporter/0.log"
Jul 14 22:39:15.899899 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:15.899869    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lght6_8d788a96-bf9b-4958-b4c5-d0ef6893b7cf/extractor/0.log"
Jul 14 22:39:16.218232 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:16.218199    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tpwxp/perf-node-gather-daemonset-v4lhf"
Jul 14 22:39:18.323551 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:18.323508    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-269c5_4b9faa79-d47a-4a43-97ea-27175eaecfc3/manager/0.log"
Jul 14 22:39:18.527816 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:18.527764    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zqm5f_62f669f2-7166-41c7-b4f7-fef4275cbfdd/manager/0.log"
Jul 14 22:39:18.829558 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:18.829531    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c99649887-j8pwd_fea1c75a-2dbd-4fe5-8520-2c4de6fa004a/manager/0.log"
Jul 14 22:39:18.856424 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:18.856332    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c99649887-j8pwd_fea1c75a-2dbd-4fe5-8520-2c4de6fa004a/copy-manifests/0.log"
Jul 14 22:39:20.331137 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:20.331074    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-h7kdq_1e8ed256-e76a-4569-8d38-c3f21ba11108/openshift-lws-operator/0.log"
Jul 14 22:39:20.490570 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:20.490544    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dcf98947-d5nvn_e4db6d7b-83b1-4af9-9684-a5d87c74d924/kube-auth-proxy/0.log"
Jul 14 22:39:20.584964 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:20.584865    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-kcssg_294cb6a0-fcdf-4a18-b372-7cf2f48ba629/payload-pre-processing/0.log"
Jul 14 22:39:22.022904 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:22.022872    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-9dcf98947-d5nvn_e4db6d7b-83b1-4af9-9684-a5d87c74d924/kube-auth-proxy/0.log"
Jul 14 22:39:22.118971 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:22.118929    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-kcssg_294cb6a0-fcdf-4a18-b372-7cf2f48ba629/payload-pre-processing/0.log"
Jul 14 22:39:25.888108 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:25.888061    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vsttd_9f9f61e4-41b3-4e1a-8542-39b4e269327a/kube-storage-version-migrator-operator/1.log"
Jul 14 22:39:25.889139 ip-10-0-130-163 kubenswrapper[2570]: I0714 22:39:25.889113    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-vsttd_9f9f61e4-41b3-4e1a-8542-39b4e269327a/kube-storage-version-migrator-operator/0.log"